This article briefly introduces databases, and how to use them with Node/Express apps. It then goes on to show how we can use Mongoose to provide database access for the LocalLibrary website. It explains how object schema and models are declared, the main field types, and basic validation. It also briefly shows a few of the main ways in which you can access
model data. Table of Contents OverviewLibrary staff will use the Local Library website to store information about books and borrowers, while library members will use it to browse and search for books, find out whether there are any copies available, and then reserve or borrow them. In order to store and retrieve information efficiently, we will store it in a database. Express apps can use many different databases, and there are several approaches you can use for performing Create, Read, Update and Delete (CRUD) operations. This tutorial provides a brief overview of some of the available options and then goes on to show in detail the particular mechanisms selected. What databases can I use?Express apps can use any database supported by Node (Express itself doesn't define any specific additional behavior/requirements for database management). There are many popular options, including PostgreSQL, MySQL, Redis, SQLite, and MongoDB. When choosing a database, you should consider things like time-to-productivity/learning curve, performance, ease of replication/backup, cost, community support, etc. While there is no single "best" database, almost any of the popular solutions should be more than acceptable for a small-to-medium-sized site like our Local Library. For more information on the options see Database integration (Express docs). What is the best way to interact with a database?There are two common approaches for interacting with a database:
The very best performance can be gained by using SQL, or whatever query language is supported by the database. ODM's are often slower because they use translation code to map between objects and the database format, which may not use the most efficient database queries (this is particularly true if the ODM supports different database backends, and must make greater compromises in terms of what database features are supported). The benefit of using an ORM is that programmers can continue to think in terms of JavaScript objects rather than database semantics — this is particularly true if you need to work with different databases (on either the same or different websites). They also provide an obvious place to perform data validation. Note: Using ODM/ORMs often results in lower costs for development and maintenance! Unless you're very familiar with the native query language or performance is paramount, you should strongly consider using an ODM. What ORM/ODM should I use?There are many ODM/ORM solutions available on the npm package manager site (check out the odm and orm tags for a subset!). A few solutions that were popular at the time of writing are:
As a general rule, you should consider both the features provided and the "community activity" (downloads, contributions, bug reports, quality of documentation, etc.) when selecting a solution. At the time of writing Mongoose is by far the most popular ODM, and is a reasonable choice if you're using MongoDB for your database. Using Mongoose and MongoDB for the LocalLibraryFor the Local Library example (and the rest of this topic) we're going to use the Mongoose ODM to access our library data. Mongoose acts as a front end to MongoDB, an open source NoSQL database that uses a document-oriented data model. A "collection" of "documents" in a MongoDB database is analogous to a "table" of "rows" in a relational database. This ODM and database combination is extremely popular in the Node community, partially because the document storage and query system looks very much like JSON, and is hence familiar to JavaScript developers. Note: You don't need to know MongoDB in order to use Mongoose, although parts of the Mongoose documentation are easier to use and understand if you are already familiar with MongoDB. The rest of this tutorial shows how to define and access the Mongoose schema and models for the LocalLibrary website example. Designing the LocalLibrary modelsBefore you jump in and start coding the models, it's worth taking a few minutes to think about what data we need to store and the relationships between the different objects. We know that we need to store information about books (title, summary, author, genre, ISBN) and that we might have multiple copies available (with globally unique ids, availability statuses, etc.). We might need to store more information about the author than just their name, and there might be multiple authors with the same or similar names. We want to be able to sort information based on the book title, author, genre, and category. When designing your models it makes sense to have separate models for every "object" (a group of related information). In this case some obvious candidates for these models are books, book instances, and authors. You might also want to use models to represent selection-list options (e.g. like a drop-down list of choices), rather than hard-coding the choices into the website itself — this is recommended when all the options aren't known up front or may change. A good example is a genre (e.g. fantasy, science fiction, etc.). Once we've decided on our models and fields, we need to think about the relationships between them. With that in mind, the UML association diagram below shows the models we'll define in this case (as boxes). As discussed above, we've created models for the book (the generic details of the book), book instance (status of specific physical copies of the book available in the system), and author. We have also
decided to have a model for the genre so that values can be created dynamically. We've decided not to have a model for the The diagram also shows the relationships between the models, including their multiplicities. The multiplicities are the numbers on the diagram showing
the numbers (maximum and minimum) of each model that may be present in the relationship. For example, the connecting line between the boxes shows that Note: As discussed in our Mongoose primer
below it is often better to have the field that defines the relationship between the documents/models in just one model (you can still find the reverse relationship by searching for the associated Note: The next section provides a basic primer explaining how models are defined and used. As you read it, consider how we will construct each of the models in the diagram above. Mongoose primerThis section provides an overview of how to connect Mongoose to a MongoDB database, how to define a schema and a model, and how to make basic queries. Installing Mongoose and MongoDBMongoose is installed in your project (package.json) like any other dependency — using npm. To install it, use the following command inside your project folder: Installing Mongoose adds all its dependencies, including the MongoDB database driver, but it does not install MongoDB itself. If you want to install a MongoDB server then you can download installers from here for various operating systems and install it locally. You can also use cloud-based MongoDB instances. Note: For this tutorial, we'll be using the MongoDB Atlas cloud-based database as a service free tier to provide the database. This is suitable for development and makes sense for the tutorial because it makes "installation" operating system independent (database-as-a-service is also one approach you might use for your production database). Connecting to MongoDBMongoose requires a connection to a MongoDB database. You can
You can get the default Note: If you need to create additional connections you can use Defining and creating modelsModels are defined using the Schemas are then "compiled" into models using the Note: Each model maps to a collection of documents in the MongoDB database. The documents will contain the fields/schema types defined in the model Defining schemasThe code fragment below shows how you might define a simple schema. First you
In the case above we just have two fields, a string and a date. In the next sections, we will show some of the other field types, validation, and other methods. Creating a modelModels are created from schemas using the
The first argument is the singular name of the collection that will be created for your model (Mongoose will create the database collection for the above model SomeModel above), and the second argument is the schema you want to use in creating the model. Note: Once you've defined your model classes you can use them to create, update, or delete records, and run queries to get all records or particular subsets of records. We'll show you how to do this in the Using models section, and when we create our views. Schema types (fields)A schema can have an arbitrary number of fields — each one represents a field in the documents stored in MongoDB. An example schema showing many of the common field types and how they are declared is shown below.
Most of the SchemaTypes (the descriptors after "type:" or after field names) are self-explanatory. The exceptions are:
The code also shows both ways of declaring a field:
For more information about options see SchemaTypes (Mongoose docs). ValidationMongoose provides built-in and custom validators, and synchronous and asynchronous validators. It allows you to specify both the acceptable range of values and the error message for validation failure in all cases. The built-in validators include:
The example below (slightly modified from the Mongoose documents) shows how you can specify some of the validator types and error messages:
For complete information on field validation see Validation (Mongoose docs). Virtual propertiesVirtual properties are document properties that you can get and set but that do not get persisted to MongoDB. The getters are useful for formatting or combining fields, while setters are useful for de-composing a single value into multiple values for storage. The example in the documentation constructs (and deconstructs) a full name virtual property from a first and last name field, which is easier and cleaner than constructing a full name every time one is used in a template. Note: We will use a virtual property in the library to define a
unique URL for each model record using a path and the record's For more information see Virtuals (Mongoose documentation). Methods and query helpersA schema can also have instance methods, static methods, and query helpers. The instance and static methods are similar, but with the obvious difference that an instance method is associated with a particular record and has access to the current object. Query helpers allow you to extend mongoose's chainable query
builder API (for example, allowing you to add a query "byName" in addition to the Using modelsOnce you've created a schema you can use it to create models. The model represents a collection of documents in the database that you can search, while the model's instances represent individual documents that you can save and retrieve. We provide a brief overview below. For more information see: Models (Mongoose docs). Creating and modifying documentsTo create a record you can define an instance of the model and then call
Creation of records (along with updates, deletes, and queries) are asynchronous operations — you supply a callback that is called when the operation completes. The API uses the error-first argument convention, so the first argument for the callback will always be an error value (or null). If the API returns some result, this will be provided as the second argument. You can also use
Every model has an associated connection (this will be the default connection when you use You can access the fields in this new record using the dot syntax, and change the values. You have to call
Searching for recordsYou can search for records using query methods, specifying the query conditions as a JSON document. The code fragment below shows how you might find all athletes in a database that play tennis, returning just the fields for athlete name and age. Here we just specify one matching field (sport) but you can add more criteria, specify regular expression criteria, or remove the conditions altogether to return all athletes.
If you specify a callback, as shown above, the query will execute immediately. The callback will be invoked when the search completes. Note: All callbacks in Mongoose use the pattern Note: It is important to remember that not finding any results is not an error for a search — but it may be a fail-case in the context of your application. If your application expects a search to find a value you can either check the result in the callback
( If you don't specify a callback then the API will return a variable of type Query. You can use this query object to build up your query and then execute it (with a callback) later using the
Above we've defined the query conditions in the
The find() method gets all matching records, but often you just want to get one match. The following methods query for a single record:
Note: There is also a There is a lot more you can do with queries. For more information see: Queries (Mongoose docs). You can create references from one document/model instance to another using the For example, the following schema defines authors and stories. Each author can have multiple stories, which we
represent as an array of
We can save our references to the related document by assigning the
Our story document now has an author referenced by the author document's ID. In order to get the author information in the story results
we use
Note: Astute readers will have noted that we added an author to our story, but we didn't do anything to add our story to our author's A better way is to get the
This is almost everything you need to know about working with related items for this tutorial. For more detailed information see Population (Mongoose docs). One schema/model per fileWhile you can create schemas and models using any file structure you like, we highly recommend defining each model schema in its own module (file), then exporting the method to create the model. This is shown below:
You can then require and use the model immediately in other files. Below we show how you might use it to get all instances of the model.
Setting up the MongoDB databaseNow that we understand something of what Mongoose can do and how we want to design our models, it's time to start work on the LocalLibrary website. The very first thing we want to do is set up a MongoDB database that we can use to store our library data. For this tutorial, we're going to use the MongoDB Atlas cloud-hosted sandbox database. This database tier is not considered suitable for production websites because it has no redundancy, but it is great for development and prototyping. We're using it here because it is free and easy to set up, and because MongoDB Atlas is a popular database as a service vendor that you might reasonably choose for your production database (other popular choices at the time of writing include Compose, ScaleGrid and ObjectRocket). Note: If you prefer you can set up a MongoDb database locally by downloading and installing the appropriate binaries for your system. The rest of the instructions in this article would be similar, except for the database URL you would specify when connecting. Note, however, that the Express Tutorial Part 7: Deploying to Production tutorial requires some form of remote database, since the free tier of the Heroku service does not provide persistent storage. It is therefore highly recommended to use MongoDB Atlas. You will first need to create an account with MongoDB Atlas (this is free, and just requires that you enter basic contact details and acknowledge their terms of service). After logging in, you'll be taken to the home screen:
You have now created the database, and have a URL (with username and password) that can be used to access it. This will look something like: Install MongooseOpen a command prompt and navigate to the directory where you created your skeleton Local Library website. Enter the following command to install Mongoose (and its dependencies) and add it to your package.json file, unless you have already done so when reading the Mongoose Primer above. Connect to MongoDBOpen /app.js (in the root of your project) and copy the following text below where you declare the Express application object
(after the line
As discussed in the Mongoose primer above, this code creates the default connection to the database and binds to the error event (so that errors will be printed to the console). Defining the LocalLibrary SchemaWe will define a separate module for each model, as discussed above. Start by creating a folder for our models in the project root (/models) and then create separate files for each of the models: /express-locallibrary-tutorial // the project root /models author.js book.js bookinstance.js genre.js Copy the
We've also declared a virtual for the AuthorSchema named "url" that returns the absolute URL required to get a particular instance of the model — we'll use the property in our templates whenever we need to get a link to a particular author. Note: Declaring our URLs as a virtual in the schema is a good idea because then the URL for an item only ever needs to be changed in one place. At this point, a link using this URL wouldn't work, because we haven't got any routes handling code for individual model instances. We'll set those up in a later article! At the end of the module, we export the model. Book model Copy the
The main difference here is that we've created two references to other models:
BookInstance model Finally, copy the
The new things we show here are the field options:
Everything else should be familiar from our previous schema. Genre model - challenge!Open your ./models/genre.js file and create a schema for storing genres (the category of book, e.g. whether it is fiction or non-fiction, romance or military history, etc.). The definition will be very similar to the other models:
Testing — create some itemsThat's it. We now have all models for the site set up! In order to test the models (and to create some example books and other items that we can use in our next articles) we'll now run an independent script to create items of each type:
Note: Go to your database on mongoDB Atlas (in the Collections tab). You should now be able to drill down into individual collections of Books, Authors, Genres and BookInstances, and check out individual documents. SummaryIn this article, we've learned a bit about databases and ORMs on
Node/Express, and a lot about how Mongoose schema and models are defined. We then used this information to design and implement Last of all, we tested our models by creating a number of instances (using a standalone script). In the next article we'll look at creating some pages to display these objects. See also
In this module |