There is no doubt that almost all applications involve data storage. However, the Express framework itself can only store data through program variables, and it does not provide data persistence. Storing data in memory alone is not a realistic scenario. Because memory itself is not suitable for large-scale data storage and the data will disappear when the service is stopped. Although we can still save data in a file, the data in a file is obviously not query-friendly. So, we’ll learn how to persist data in Express in the form of a MongoDB database.
The main contents of this article are:
- How MongoDB works.
- How to use Mongoose.
- How to create user accounts safely.
- How to use the user password to perform authorization operations?
Why MongoDB?
For Web applications, the selection of databases can generally be divided into two categories: relational and non-relational. The former has the advantage of being a spreadsheet, where the data is structured and accompanied by strict rules. Typical relational databases include MySQL, SQL Server, and PostgreSQL. The latter is often referred to as a NoSQL database, and its structure is relatively flexible, which is very similar to JS.
But why are Node developers so drawn to the Mongo database in NoSQL and the popular MEAN stack?
The first reason is that Mongo is one of the most popular types of NoSQL data. This also makes the web a rich source of information about Mogon, covering all the possible pitfalls you might encounter in actual use. And as a mature project, Mongo has been recognized and applied by large companies.
Another reason is that Mongo itself is very reliable and distinctive. Its use of high-performance C++ for the underlying implementation has also earned it a lot of user trust.
Although Mongo is not implemented in JavaScript, the native shell uses the JavaScript language. This means you can manipulate Mongo from the console using JavaScript. In addition, it reduces the cost of learning a new language for Node developers.
Of course, Mongo is not the right choice for all Express applications, and relational databases still play a very important role. By the way, CouchDB in NoSQL is also very powerful.
Note: although this article will only introduce the use of Mongo and Mongoose class libraries. But if you’re as familiar with SQL as I am and want to use relational databases in Express, you can check out Sequelize. It provides good support for many relational databases.
How does Mongo work
Before we use Mongo, let’s take a look at how Mongo works.
Most applications use a database like Mongo on the server for persistence. Although you can create multiple databases in an application, most will only use one.
If you want to access these databases properly, you first need to run a Mongo service. The client sends instructions to the server to implement various operations on the database. Programs that connect the client and server are usually called database drivers. For Mongo database, its database driver in Node environment is Mongoose.
Each database has one or more arrays of data. For example, a simple blog application might have a collection of articles, a collection of users. But these data sets are far more powerful than arrays. For example, you can query a collection of users over the age of 18.
And each collection stores documents in JSON form, even though it’s not technically JSON. Each document corresponds to a record, and each record contains several field attributes. In addition, document records in the same collection do not necessarily have the same field attributes. This is one of the biggest differences between NoSQL and relational databases.
In fact, the document technically uses Binary JSON, called BSON for short. We don’t play with BSON directly during actual code writing. In most cases, it will be converted to JavaScript objects. In addition, the encoding and decoding of BSON is different from that of JSON. BSON also supports more types, such as date and time stamps. The following figure shows the database usage structure in the application:
Last but not least, Mongo adds an _id attribute to each document record to indicate that the record is unique. If two document records of the same type have the same ID attribute, then it can be inferred that they are the same record.
Issues for SQL users to be aware of
If you have a background in relational databases, you will actually find that many of Mongo’s concepts correspond to SQL meanings.
First, the concept of a document in Mongo is essentially the equivalent of a row of records in SQL. In the application’s user system, each user is a document in Mongo and a record in SQL. But unlike SQL, Mongo does not have a mandatory schema at the database level, so a user record without a user name or email address is legal in Mongo.
Second, collections in Mongo correspond to tables in SQL that are used to store the same type of record.
Similarly, databases in Mongo are very similar in concept to SQL databases. Typically, an application has only one database, and a database can contain multiple collections or tables.
For a more detailed glossary of terms, see the official document.
Mongo environment setup
Before use, the first task is of course to install the Mongo database on the machine and pull up the service. If you are running macOS and don’t like the command line mode, you can set up the environment by installing the Mongo.app. The brew install mongodb command can be used if you are familiar with command-line interaction.
Ubuntu can refer to the documentation, while Debian can refer to the documentation for Mongo installation.
In addition, throughout this book we will assume that your installation is using the default configuration of the Mongo database. This means that you did not change the Mongo service port number and used the default 27017.
Use Mongoose to operate Mongo database
With Mongo installed, the next problem is how to manipulate the database in the Node environment. The best way to do this is to use the official Mongoose library. Its official documentation describes it as:
Mongoose provides an intuitive, schema-based solution to common database problems such as data modeling, type conversion, and data validation.
In other words, Mongoose provides more than just a bridge between Node and Mongo. Let’s familiarize ourselves with Mongoose’s features by building a simple web site with a user system.
The preparatory work
In order to better understand the content of this article, we will develop a simple social application. The app will be able to register users, edit personal information and browse others’ information. Here we call it “Learn About Me” or “LAM” for short. The application contains the following pages:
- Home page, which lists all users and can be clicked to view user details.
- Personal information page, used to display user name and other information.
- User registration page.
- User login page.
As before, first we need to create a new project directory and edit the information in the package.json file:
{
"name": "learn-about-me"."private": true."scripts": {
"start": "node app"
},
"dependencies": {
"bcrypt-nodejs": "0.0.3"."body-parser": "^ 1.6.5"."connect-flash": ^ "while"."cookie-parser": "^ 1.3.2." "."ejs": "^ 1.0.0"."express": "^ 4.0.0"."express-session": "^ 1.7.6." "."mongoose": "^ 3.8.15"."passport": "^ 0.2.0." "."passport-local": "^ 1.0.0"}}Copy the codeNext, run NPM Install to install these dependencies. The role of each of these dependencies will be described later.
It should be noted that here we introduce a pure JS implementation of the encryption module bcrypt-nodejs. In fact, there is a USE of C language in NPM encryption module Bcrypt. Although bcrypt performs better, it is not as easy to install as bcrypt-nodejs because it requires compiling C code. However, the two libraries are equally functional and can be switched freely.
Creating the User model
As mentioned earlier, Mongo stores data in BSON form. For example, the BSON representation of Hello World is:
\x16\x00\x00\x00\x02hello\x00\x06\x00\x00\x00world\x00\x00Copy the codeAlthough computers are perfectly capable of understanding the BSON format, it is clear that BSON is not an easy-to-read format for humans. As a result, developers invented the concept of a database model that was easier to understand. The database model defines database objects in a manner that approximates human language. A model represents a database record and often an object in a programming language. For example, here it represents a JavaScript object.
In addition to representing a record of a database, models are usually accompanied by methods of data validation, data extension, and so on. Here’s a look at some of these properties in Mongoose.
In the example, we will create a user model with the following attributes:
- User name. This attribute cannot be default and must be unique.
- Password, also cannot default.
- Creation time.
- User nickname, used for information display and optional.
- Personal profile, not required.
In Mongoose we used schema to define the user model. In addition to including the above attributes, you will later add some type methods to them. Create the Models folder at the root of your project, then create a file named user.js in it and copy the following code:
var mongoose = require("mongoose");
var userSchema = mongoose.Schema({
username: { type: String, require: true, unique: true },
password: { type: String, require: true },
createdAt: {type: Date, default: Date.now },
displayName: String,
bio: String
});Copy the codeFrom the above code, we can see that the definition of the property field is very simple. At the same time, we also made a convention on the data type, uniqueness, default and default value of the field.
Once the model is defined, the next step is to define the methods in the model. First, let’s add a simple method that returns the user name. Return the nickname if the user has defined a nickname otherwise return the user name. The code is as follows:
. userSchema.methods.name =function() {
return this.displayName || this.username;
}Copy the codeSimilarly, to ensure the security of user information in the database, password fields must be stored in ciphertext. In this way, even if there is database leakage or invasion behavior, user information security can be ensured to a certain extent. Here we will use the Bcrypt program to do a one-way hash hash of the user password and store the encrypted result in the database.
First, we need to introduce the Bcrypt class library in the header of the user.js file. In the process of using we can increase the number of hashes to improve data security. Of course, the hash operation is very manipulative, so we should choose a relatively moderate number. For example, in the following code we set the hash count to 10.
var bcrypt = require("bcrypt-nodejs");
var SALT_FACTOR = 10;Copy the codeOf course, the password should be hashed before the data is saved. So this part of the code should be done in the callback function before saving the data, like this:
. var noop =function() {}; // Save the callback before the operation userschema.pre ("save".function(done) {
var user = this;
if(! user.isModified("password")) {
return done(a); } bcrypt.genSalt(SALT_FACTOR,function(err, salt) {
if (err) {
return done(err);
}
bcrypt.hash(user.password, salt, noop,
function(err, hashedPassword) {
if (err) {
return done(err);
}
user.password = hashedPassword;
done();
}
);
});
});Copy the codeThis callback function is called before each database save, so it ensures that your password is saved in ciphertext.
In addition to encrypting passwords, another common requirement is user authorization. For example, password authentication during user login operations.
. userSchema.methods.checkPassword =function(guess, done) {
bcrypt.compare(guess, this.password, function(err, isMatch) {
done(err, isMatch);
});
}Copy the codeFor security reasons, we use the bcrypt.compare function here rather than the simple equality judgment ===.
With the model definition and common method implementation complete, we then need to expose it for other code to use. But exposing the model is very simple with two lines of code:
. var User = mongoose.model("User", userSchema);
module.exports = User;Copy the codeThe complete code in the models/user.js file is as follows:
Var bcrypt = require("bcrypt-nodejs");
var SALT_FACTOR = 10;
var mongoose = require("mongoose");
var userSchema = mongose.Schema({
username: { type: String, require: true, unique: true },
password: { type: String, require: true },
createdAt: {type: Date, default: Date.now },
displayName: String,
bio: String
});
userSchema.methods.name = function() {
return this.displayName || this.username;
}
var noop = function() {};
userSchema.pre("save".function(done) {
var user = this;
if(! user.isModified("password")) {
return done(a); } bcrypt.genSalt(SALT_FACTOR,function(err, salt) {
if (err) { return done(err); }
bcrypt.hash(user.password, salt, noop,
function(err, hashedPassword) {
if (err) { return done(err); }
user.password = hashedPassword;
done();
}
);
});
});
userSchema.methods.checkPassword = function(guess, done) {
bcrypt.compare(guess, this.password, function(err, isMatch) {
done(err, isMatch);
});
}
var User = mongoose.model("User", userSchema);
module.exports = User;Copy the codeModel USES
Once the model is defined, the next step is to use it on the home page, edit page, register page, and so on. The process is relatively simple compared to the previous model definition.
First, create the main entry file app.js in the project root directory and copy the following code:
var express = require("express");
var mongoose = require("mongoose");
var path = require("path");
var bodyParser = require("body-parser");
var cookieParser = require("cookie-parser");
var session = require("express-session");
var flash = require("connect-flash");
var routes = require("./routes"); var app = express(); // Connect to your MongoDB servertestDatabase mongoose. Connect ("mongodb://localhost:27017/test");
app.set("port", process.env.PORT || 3000);
app.set("views", path.join(__dirname, "views"));
app.set("view engine"."ejs");
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
app.use(session({
secret: "TKRv0IJs=HYqrvagQ#&! F! %V]Ww/4KiVs$s,<<MX",
resave: true,
saveUninitialized: true
}));
app.use(flash());
app.use(routes);
app.listen(app.get("port"), function() {
console.log("Server started on port " + app.get("port"));
});Copy the codeNext, we need to implement the routing middleware used above. Create routes.js in the root directory and copy the code:
var express = require("express");
var User = require("./models/user");
var router = express.Router();
router.use(function(req, res, next) {
res.locals.currentUser = req.user;
res.locals.errors = req.flash("error");
res.locals.infos = req.flash("info");
next();
});
router.get("/".function(req, res, next) {
User.find()
.sort({ createdAt: "descending" })
.exec(function(err, users) {
if (err) { return next(err); }
res.render("index", { users: users });
});
});
module.exports = router;Copy the codeIn these two pieces of code, first of all, we used Mongoose to connect the database. The list of users is then asynchronously retrieved through user.find in the routing middleware and passed to the home view template.
Next, we turn to the implementation of the home view. First create the views folder in the root directory, then add the first template file _header.ejs to the folder:
<! DOCTYPE html> <html lang="en">
<head>
<meta charset="UTF-8">
<title>Learn About Me</title>
<link rel="stylesheet" href="/ / maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css">
</head>
<body>
<div class="navbar navbar-default navbar-static-top" role="navigation">
<div class="container">
<div class="navbar-header">
<a class="navbar-brand" href="/">Learn About Me</a> </div> <! Change the navigation bar if the user is already logged in. CurrentUser does not exist in your code initially, so it will always display a status --> <ul class="nav navbar-nav navbar-right"> < %if (currentUser) { %>
<li>
<a href="/edit">
Hello, <%= currentUser.name() %>
</a>
</li>
<li><a href="/logout">Log out</a></li>
<% } else { %>
<li><a href="/login">Log in</a></li>
<li><a href="/signup">Sign up</a></li>
<% } %>
</ul>
</div>
</div>
<div class="container">
<% errors.forEach(function(error) { %>
<div class="alert alert-danger" role="alert">
<%= error %>
</div>
<% }) %>
<% infos.forEach(function(info) { %>
<div class="alert alert-info" role="alert">
<%= info %>
</div>
<% }) %>Copy the codeYou may have noticed that the names of these files start with an underscore. This is a community convention and all component templates are distinguished by an underscore.
Next, add the second generic component template _footer.js:
</div>
</body>
</html>Copy the codeFinally, we add the home view template file. The view template takes the Users variable passed into the middleware and completes the rendering:
<% include _header %> <h1>Welcome to Learn About Me! </h1> <% users.forEach(function(user) { %>
<div class="panel panel-default">
<div class="panel-heading">
<a href="/users/<%= user.username %>">
<%= user.name() %>
</a>
</div>
<% if (user.bio) { %>
<div class="panel-body"><%= user.bio %></div>
<% } %>
</div>
<% }) %>
<% include _footer %>Copy the codeAfter ensuring that the code is correct, start the Mongo database service and pull up the project using NPM start. Then, go to localhost:3000 in your browser and you’ll see the home page shown below:
Of course, there’s nothing in the database at this point so there’s no user information there.
Next, we will implement the user user registration and login functions. Before we do that, we need to introduce the Body-parser module in app.js and use it to parse subsequent request parameters.
var bodyParser = require("body-parser"); . app.use(bodyParser.urlencoded({ extended:false})); ...Copy the codeTo improve security, here we set extended to false for the Body-Parser module. Next, we add the middleware handler for sign-up to routes.js:
var passport = require("passport"); . router.get("/signup".function(req, res) {
res.render("signup");
});
router.post("/signup".function(req, res, next) {var username = req.body.username; var password = req.body.password; // Calling findOne returns only one user. FindOne ({username: username},function(err, user) {
if (err) { returnnext(err); } // Check whether the user existsif (user) {
req.flash("error"."User already exists");
return res.redirect("/signup"); } var newUser = newUser ({username: username, password: password}); // Insert record newuser.save (next); }); // Perform login and redirect}, passport. Authenticate ("login", {
successRedirect: "/",
failureRedirect: "/signup",
failureFlash: true
}));Copy the codeAfter defining the routing middleware, let’s implement the view template signup. Ejs file.
Ejs <% include _header %> <h1>Sign up</h1> <form action="/signup" method="post">
<input name="username" type="text" class="form-control" placeholder="Username" required autofocus>
<input name="password" type="password" class="form-control" placeholder="Password" required>
<input type="submit" value="Sign up" class="btn btn-primary btn-block">
</form>
<% include _footer %>Copy the codeIf you successfully create a user and visit the home page again, you will see a list of users:
The UI of the registration page is roughly as follows:
Before realizing the login function, we first complete the personal information display function. Add the following middleware functions to routes.js:
. router.get("/users/:username".function(req, res, next) {
User.findOne({ username: req.params.username }, function(err, user) {
if (err) { return next(err); }
if(! user) {return next(404); }
res.render("profile", { user: user }); }); }); .Copy the codeNext, write the view template file profile.ejs:
<% include _header %> <! Use currentUser to determine your login status. But now it always will befalseState - > < %if ((currentUser) && (currentUser.id === user.id)) { %>
<a href="/edit" class="pull-right">Edit your profile</a>
<% } %>
<h1><%= user.name() %></h1>
<h2>Joined on <%= user.createdAt %></h2>
<% if (user.bio) { %>
<p><%= user.bio %></p>
<% } %>
<% include _footer %>Copy the codeIf you now go to the user details page from the home page, you should see something like the following:
Use the Passport to authenticate the user
In addition to these basic functions, the User model does important things like login and permission authentication. This is the biggest difference between the User model and other models. So the next task is to implement the login page and authenticate password and permissions.
To save a lot of unnecessary work, we’ll use the third-party Passport module here. This template is Node middleware specifically designed to handle requests for validation. With this middleware, complex authentication operations can be implemented with only a small piece of code. Passport does not specify how to authenticate a user, however, it just provides modular functions.
Set up the Passport
The setup process for Passport involves three things:
- Set up the Passport middleware.
- Set up Passport’s serialization and deserialization of the User model.
- Tell Passport how to authenticate the User.
First, when initializing the Passport environment, you need to introduce some other middleware into your project. They are:
- body-parser
- cookie-parser
- express-session
- connect-flash
- passport.initialize
- passport.session
The first four of these middleware have already been introduced. Body-parser is used for parameter parsing. Cookie-parser handles cookies retrieved from browsers; Express-session is used to process user sessions. Connect-flash displays error messages to the user.
Finally, we need to introduce the Passport module in app.js and then call two of the middleware functions.
var bodyParser = require("body-parser");
var cookieParser = require("cookie-parser");
var flash = require("connect-flash");
var passport = require("passport");
var session = require("express-session"); . app.use(bodyParser.urlencoded({ extended:false})); app.use(cookieParser()); App.use (session({// a random sequence of letters is required. Strings do not need to be like this:"TKRv0IJs=HYqrvagQ#&! F! %V]Ww/4KiVs$s,<<MX",
resave: true,
saveUninitialized: true})); app.use(flash()); app.use(passport.initialize()); app.use(passport.session()); .Copy the codeIn the code, we use a random string to encode the client’s session. This can increase the security of cookies to a certain extent. Setting resave to true ensures that the session is refreshed even if it has not been modified.
The next step is to set up Passport’s serialization and deserialization of the User model. This allows the Passport to convert session and User objects to each other. The Passport documentation describes this operation as follows:
In a standard Web application, users need to be authenticated only when the client sends a login request. If the authentication succeeds, a session is created between the two and stored in a cookie for maintenance. Any subsequent operations do not authenticate, but instead use the session specified uniquely in the cookie. Therefore, Passport needs to convert session and User objects to each other through serialization and deserialization.
In order to maintain the code later, we will create a new file named setuppassport.js and put the serialized and deserialized code into it. Finally, we introduce it into app.js:
... varsetUpPassport = require("./setuppassport"); ... var app = express(); mongoose.connect("mongodb://localhost:27017/test");
setUpPassport(); ...Copy the codeHere is the code in setuppassport.js. Since the User object has an ID attribute as a unique identifier, we use it to serialize and deserialize the User object:
Var passport = require(setuppassport.js)"passport");
var User = require("./models/user");
module.exports = function() {
passport.serializeUser(function(user, done) {
done(null, user._id);
});
passport.deserializeUser(function(id, done) {
User.findById(id, function(err, user) {
done(err, user);
});
});
}Copy the codeNow comes the hard part, how do you authenticate?
Before you can begin authentication, there is one more small task to complete: setting up authentication policies. The Passport comes with Facebook and Google authentication policies, but here we need to set it to a Local strategy. Because the rules and code for the validation part are implemented by ourselves.
First, we introduce LocalStrategy in setuppassport.js
. var LocalStrategy = require("passport-local").Strategy; ...Copy the codeNext, follow these steps to use LocalStrategy for specific validation:
- None Example Query the user.
- If the user does not exist, the user fails to pass the authentication.
- If the user exists, the password is compared. If the match is successful, the system returns the current user; otherwise, the system displays “Incorrect password”.
Here’s how to translate these steps into concrete code:
// setuppassport.js validation code... passport.use("login", new LocalStrategy(function(username, password, done) {
User.findOne({ username: username }, function(err, user) {
if(err) { return done(err); }
if(! user) {return done(null, false, { message: "No user has that username!" });
}
user.checkPassword(password, function(err, isMatch) {
if (err) { return done(err); }
if (isMatch) {
return done(null, user);
} else {
return done(null, false, { message: "Invalid password."}); }}); }); })); .Copy the codeOnce the policy definition is complete, it can then be invoked anywhere in the project.
Finally, we need to complete some views and functions:
- The login
- logout
- Edit personal information after login
First, we implement the login interface view. Add login routing middleware to routes.js:
. router.get("/login".function(req, res) {
res.render("login"); }); .Copy the codeIn the login view login.ejs, we receive a user name and a password and send a POST request for login:
<% include _header %>
<h1>Log in</h1>
<form action="/login" method="post">
<input name="username" type="text" class="form-control" placeholder="Username" required autofocus>
<input name="password" type="password" class="form-control" placeholder="Password" required>
<input type="submit" value="Log in" class="btn btn-primary btn-block">
</form>
<% include _footer %>Copy the codeNext, we need to process the POST request. The Passport’s authentication function is used.
// routes.js var passport = require("passport"); . router.post("/login", passport.authenticate("login", {
successRedirect: "/",
failureRedirect: "/login",
failureFlash: true})); .Copy the codeThe passport. Authenticate function returns a callback. This function redirects the different validation results depending on what we specify. For example, a successful login redirects to the home page, and a failure redirects to the login page.
The logout operation is relatively simple, and the code is as follows
// routes.js logout section... router.get("/logout".function(req, res) {
req.logout();
res.redirect("/"); }); .Copy the codePassport also comes with req.user and connect-Flash information. Go back to the previous code and you’ll get a better idea.
. router.use(function(req, res, next) {// Set a few useful variables for your template res.locals.currentUser = req.user; res.locals.errors = req.flash("error");
res.locals.infos = req.flash("info"); next(); }); .Copy the codeLogin and logout play smoke, the following is the turn of personal information editing function.
First, let’s implement a generic middleware tool called ensureAuthenticated. This middleware function checks the permissions of the current user and redirects to the login page if the check fails.
// routes.js ensureAuthenticated middleware...functionEnsureAuthenticated (req, res, next) {// A function provided by Passportif (req.isAuthenticated()) {
next();
} else {
req.flash("info"."You must be logged in to see this page.");
res.redirect("/login"); }}...Copy the codeNext, we call this function in the editing middleware. Because we need to make sure that the current user has edit rights before we start editing.
// GET /edit (in router.js)... // Ensure that the user is authenticated; Run your request handler router.get() if they are not redirected"/edit", ensureAuthenticated, function(req, res) {
res.render("edit"); }); .Copy the codeNext we need to implement the Edit. ejs view template file. The content of the view template is very simple, containing only changes to the user’s nickname and profile.
// views/edit.ejs
<% include _header %>
<h1>Edit your profile</h1>
<form action="/edit" method="post">
<input name="displayname" type="text" class="form-control" placeholder="Display name" value="<%= currentUser.displayName || "" %>">
<textarea name="bio" class="form-control" placeholder="Tell us about yourself!"> <%= currentUser.bio || "" %></textarea>
<input type="submit" value="Update" class="btn btn-primary btn-block">
</form>
<% include _footer %>Copy the codeFinally, we need to process the request submitted after the modification. Permission authentication is also required before database updates.
// POST /edit (routes.js)... // Normally, this would be a PUT request, but HTML forms only support GET and POST router.post("/edit", ensureAuthenticated, function(req, res, next) {
req.user.displayName = req.body.displayname;
req.user.bio = req.body.bio;
req.user.save(function(err) {
if (err) {
next(err);
return;
}
req.flash("info"."Profile updated!");
res.redirect("/edit"); }); }); .Copy the codeThis code simply updates the fields of the corresponding database record. The final rendered edit view looks like this:
Finally, you can create some test data to verify all the features of the sample application.
conclusion
The contents of this article are:
- How Mongo works.
- Use of Mongoose.
- Use bcrypt to encrypt specific fields to improve data security.
- Use Passport for permission authentication.
The original address