Continuous integration and deployment of NodeJS applications based on Travis CI + PM2

preface

I found that coding was already a headache once I had more projects on hand and as project complexity increased, but deploying to production was even more taxing 😵.

When I was an intern in a company, I had a project to take screenshots based on user input url, including React application and Node application.

Deploying React applications is easy. Simply place the built dist directory on the server using SCP.

Node applications are more complex:

Since it is written in TS, the post-build dist directory also needs to be placed on the server
Create a directory in the root directory and run the chmod command to change the permission to temporarily place the snapshot
Update the NPM package
Restart the PM2 (Node Process Management Tool)

At the beginning of the project, version iteration was very fast. I had to repeat the above steps several times every day. Waste Time!

In addition, in the standard development process, we also need to introduce unit testing, coverage reporting, code style checking… , and deploy the application to the server in different environments (development, testing, production), this is undoubtedly a tedious work, in line with the core idea that the front-end of operation and maintenance is not a good full stack, I urgently need to free my hands.

There is a link to the source code at the end

CI & CD

The so-called predecessors plant trees for future generations to enjoy the shade, my appeal has long been defined as two terms in the field of development:

Continuous Integration (CI)
Continuous Deployment, CD for short

It sounds lofty, but I’ll try to explain it with a picture:

An iteration of a complete project needs to go through: coding ➡️ packaging build ➡️ testing ➡️ the new code and the original code integrate correctly.

This process is called integration, and continuous integration emphasizes doing this without human intervention as soon as developers submit new code (Git push).

Similarly, continuous deployment adds another step to continuous integration: automatic deployment of applications to a specified environment (server).

Imagine if the CI/CD service automates the above steps according to your preset command after you submit your code.

In order to improve the efficiency of software development, it is necessary for us to use CI/CD, and the well-known CI/CD services in the market include Jenkins and GitLab, but they cost a lot to use.

I would recommend Travis CI, which is tied to projects on Github and automatically grabs new code whenever it comes in. Then, provide a run environment, perform tests, complete builds, and deploy to the server.

The preparatory work

To make sure you can get through the hands-on section, do the following:

A remote Linux server for deployment
Log in to Travis CI using the Github account and have Travis listen for Github repository updates

I would build a NodeJS application for API services from scratch, introduce unit tests and ESLint, and eventually implement CI/CD.

Here’s the whole idea:

Create a Github repository to store NodeJS application code
The Github repository that is being monitored receives a git push command that triggers Travis CI to build (YARN Run test, YARN Run ESlint).
When Travis CI builds correctly, call PM2’s deploy command to have the remote server pull the latest Github repository code, install dependencies, and automatically restart the service

The local server, GitHub, and remote (deployment) server can communicate with each other

For simplicity, I’ll call the local machine Local and the remote server Remote

All things are difficult before they are easy. First, realize the following two things:

PM2 Deployment requires that Remote be able to pull the Github repository code
If Travis CI has a need to access remote, make sure local has access to remote

Since Travis CI and PM2 Deployment do not provide an interactive interface at run time, they will only execute one by one according to preset script commands, which will get stuck when you need to enter a password, so we need SSH non-secret login to achieve the following relationship:

local => GitHub;
remote= > GitHub;
local= > remote;
Copy the code

The local connection making

Generate the SSH key private key public key pair and press Enter without passphrase.

$ ssh-keygen -t rsa -b 4096 -C "<your-github-email>"
Copy the code

In the local ~/. SSH directory, the following files are generated:

├ ─ ─ id_rsa ├ ─ ─ id_rsa. PubCopy the code

Id_rsa is the private key file, which stands for 🔑. Id_rsa. pub is a public key file, which stands for 🔒.

Only the private key can open the public key.

Open github.com/settings/ke… Click New SSH key to copy the contents of id_rsa.pub.

Select one of your repositories and click Clone or Download && Clone with SSH.

The local connection remote

By default, the ssh-copy-id command copies the generated public key id_rsa.pub to remote.

⚠️ switch to your own remote IP.

$SSH - copy - id [email protected]Copy the code

❗️ If the Win system cannot recognize this command, run git bash.

SSH in the remote directory. The contents in id_rsa.pub are the same as those in authorized_keys.

├ ─ ─ authorized_keysCopy the code

Try to connect to remote.

$SSH [email protected]Copy the code

If you do not need to enter the password, the SSH connection between local and remote is implemented.

Remote connection making

Connect to GitHub (local); since we have stored the public key on GitHub, we only need to upload the private key: ID_RSA to remote.

After the upload, the following files exist in the remote ~/. SSH directory:

├ ─ ─ authorized_keys ├ ─ ─ id_rsaCopy the code

Similarly, you can try using Clone with SSH to download the GitHub repository on remote to verify the connection.

So far, we have achieved SSH communication among the three.

Build the NodeJS application

Create a new repository on GitHub and Clone it locally.

Since the application implements API services based on the KOA framework, some initial configurations are performed:

$ yarn init -y
$ yarn add koa
Copy the code

For subsequent coding, you should have the following directories:

├ ─ ─ lib │ ├ ─ ─ app. Js ├ ─ ─ for server js ├ ─ ─ package. The json └ ─ ─ yarn. The lockCopy the code

Start writing code:

// lib/app.js
const Koa = require("koa");

const app = new Koa();

app.use(ctx= > {
  if (ctx.method == "GET" && ctx.path == "/user") {
    ctx.body = "hello, friend"; }});module.exports = app;
Copy the code

// server.js
const app = require("./lib/app");

app.listen("8888", () = > {console.log("server is running at http://localhost:8888");
});
Copy the code

Start the Node application:

$ node server.js
Copy the code

I created a very simple API service, when users visit http://localhost:8888/user, return “hello, friend.”

The use of Travis CI

To do this, you need to create the Travis CI configuration file and create.travis. Yml in the root directory

# Build the environment
language: node_js
# node_js version
node_js:
  - 12
after_success:
  - echo 'I successfully done'
Copy the code

⚠️ Travis CI executes the install and script life cycles by default, even if they are not explicitly defined in the configuration file.

In the case of the current configuration file, Travis CI will execute install ➡️ script ➡️ after_success after starting the build.

Building a JavaScript and Node.js project:

Install will execute by defaultnpm install
Script is executed by defaultnpm test

In addition, if Travis CI detects yarn.lock, the replacement commands are yarn and yarn test, respectively.

So, we also need to provide the test script to add to package.json:

"scripts": {
  "test": "echo just test it",
},
Copy the code

Finally, make sure that you are listening for the repository in/Account/Repositories.

All you need to do is push the modified code to the remote repository to trigger the Travis CI.

$ git push
Copy the code

Go to the Travis – CI/Dashboard, select Travis – Test in the Active Repositories panel, and see the following information:

Check the log information below. I have marked the key points with words:

Continuous integration has run, but does it feel like something is missing? Yes, the command to access remote has not been added.

Since Travis CI essentially turns on a virtualization container to perform the entire build process, it is necessary to pass it the private key: ID_RSA to support remote’s SSH connection. Id_rsa = id_rsa; id_rsa = id_rsa;

Travis CI has this in mind and provides an encryption scheme for private keys.

To encrypt the private key file, you need to use Travis, a command line tool, which is a Ruby package installed using gem:

$ gem install travis
$ travis login
Copy the code

If you fail to install Travis, check out github.com/travis-ci/t… .

After logging in successfully, use Travis encrypt-file to encrypt:

$ travis encrypt-file ~/.ssh/id_rsa --add
# Detected repository as B2D1/travis-test, is this correct? |yes| yes
# Overwrite the config file /root/travis-test/.travis.yml with the content below? (y/N) y

# Make sure to add id_rsa.enc to the git repository.
# Make sure not to add /root/.ssh/id_rsa to the git repository.
# Commit all changes to your .travis.yml.
Copy the code

After executing the command above, a decryption command is generated and added to.travis. Yml:

before_install:
- openssl aes-256-cbc -K $encrypted_9b2d7e19d83c_key -iv $encrypted_9b2d7e19d83c_iv
  -in id_rsa.enc -out ~/.ssh/id_rsa -d
Copy the code

And ❗️, be sure to copy the encrypted ID_rsa. enc to the repository, and be sure not to copy the unencrypted ID_RSA to the repository.

-out ~/.ssh/id_rsa -d -out ~/.ssh/id_rsa -d -out ~/.ssh/id_rsa -d -out ~/.ssh/id_rsa -d

The before_install phase occurs before the install phase, and this code means: Encrypted_9b2d7e19d83c_iv and ENCRYPted_9B2D7E19D83C_key are used to decrypt id_rsa.enc in the warehouse and generate the private key id_RSA in the ~/. SSH directory of the virtual container

You can find this pair of environment variables in your repository at travis-ci.org ➡️ ➡️ More options ➡️ Settings:

The connection to remote is basically complete, but there are still some holes to fill:

Lower the id_RSA file permissions; otherwise, SSH will refuse to read the secret key for security reasons
Add remote IP to the trust list of the Travis CI virtual container, otherwise remote will be asked if it is trusted when connecting to it

The changed.travis. Yml configuration is as follows:

# Build the environment
language: node_js
# node_js version
node_js:
  - 12
Add the remote server to the trust list
addons:
  ssh_known_hosts: 47.10687.3.
# decrypt id_rsa.enc and change the id_RSA permission
before_install:
  - openssl aes-256-cbc -K $encrypted_9b2d7e19d83c_key -iv $encrypted_9b2d7e19d83c_iv
    -in id_rsa.enc -out ~/.ssh/id_rsa -d
  - chmod 600 ~/.ssh/id_rsa
Connect to the remote server and print the system version
after_success:
  - ssh [email protected] 'cat /etc/issue'
Copy the code

Commit the code (git push) to see the build result:

Successfully printed version information for my remote server:

Adding unit tests

In the previous section, we simply used the echo command to quickly pass the YARN test.

It is now time to formally add unit tests to the NodeJS application. It is recommended to choose Jest + SuperTest.

Jest is an open source JavaScript testing framework for Facebook that automatically integrates assertions, JSDom, coverage reporting, and all the testing tools developers need. It is a nearly zero-configuration testing framework.

SuperTest: HTTP assertions made easy via superagent.

Install NPM package:

$ yarn add jest supertest --dev
Copy the code

Change the package. The json:

"scripts": {
  "test": "jest"
},
Copy the code

Create a new __test__/app.test.js file under the root directory and write the test code:

const app = require(".. /lib/app");
const supertest = require("supertest");
const server = app.listen();
const request = supertest(server);

test("GET /user".async done => {
  const res = await request.get("/user");
  expect(res.status).toBe(200);
  expect(res.text).toBe("hello, friend");
  done();
});

afterAll(done= > {
  server.close();
  done();
});
Copy the code

Execute the test script:

$ yarn test
Copy the code

Pass the test:

Coverage reports can also be provided with the –coverage parameter:

Add ESlint

In this section, continue refining the NodeJS application by adding ESlint to it.

ESLint is a plug-in and configurable checker for JavaScript syntax rules and code styles. ESLint can help you write high quality JavaScript code easily

Install NPM package:

$ yarn add eslint eslint-config-google --dev
Copy the code

Change the package. The json:

"scripts": {
  "lint": "eslint .",
  "test": "jest",
  "pretest": "yarn run lint"
},
Copy the code

❗️ The pretest script is automatically executed before YARN test.

Create the configuration file.eslintrc.json in the root directory

{
  "extends": ["eslint:recommended"."google"]."env": {
    "node": true
  },
  "parserOptions": {
    "ecmaVersion": 6
  },
  "rules": {
    "eqeqeq": 2
  },
  "ignorePatterns": ["ecosystem.config.js"."__tests__"]}Copy the code

The default Lint rule is adopted here: recommended & Google.

Add a new rule: the existence of non-strict equality (==) causes the program to exit (0 for close, 1 for warning, 2 for error).

Other configuration items are: setting the code environment, ECMA version, and specifying which files are not checked.

Run the lint command:

$ yarn run lint
Copy the code

The following error occurred:

You can try running the yarn run lint –fix command and ESlint will automatically fix the error. For those that cannot be automatically repaired, manually modify them.

Use the PM2 Deployment

After the above steps, CI (continuous integration) has been implemented based on Travis CI.

Only one final step remains: deploying the NodeJS application to a remote server.

Referring to the official PM2 Deployment document, all we need to do is create a configuration file and PM2 does the rest.

Create server.config.js in the root directory

module.exports = {
  apps: [{// PM2 application name
      name: "travis-test-deploy".// node starts the file
      script: "server.js"],},deploy: {
    // "prod" is the environment name
    prod: {
      // Private key directory
      key: "~/.ssh/id_rsa".// Login user
      user: "root".// Remote server
      host: ["47.106.87.3"].// Automatically adds Github to the remote server's trust list
      ssh_options: "StrictHostKeyChecking=no"./ / git branch
      ref: "origin/master".// Git repository address (SSH)
      repo: "[email protected]:B2D1/travis-test.git".// Where the project is stored on the remote server
      path: "/root/travis-test-deploy".PM2 pulls the latest branch, installs the NPM package, and starts (restarts) the NodeJS application
      "post-deploy":
        "source ~/.nvm/nvm.sh && yarn install && pm2 startOrRestart ecosystem.config.js",}}};Copy the code

Also modify.travis. Yml

# Build the environment
language: node_js
# node_js version
node_js:
  - 12
Add the remote server to the trust list
addons:
  ssh_known_hosts: 47.10687.3.
# decrypt id_rsa.enc and change the id_RSA permission
before_install:
  - openssl aes-256-cbc -K $encrypted_9b2d7e19d83c_key -iv $encrypted_9b2d7e19d83c_iv
    -in id_rsa.enc -out ~/.ssh/id_rsa -d
  - chmod 600 ~/.ssh/id_rsa
# PM2 deploy
after_success:
  - npm i -g pm2 && pm2 deploy ecosystem.config.js prod update
Copy the code

⚠️ On the first deployment, we need to initialize the project on a remote server.

$ pm2 deploy ecosystem.config.js prod setup
Copy the code

❗️ If the Win system fails, use git bash.

Then commit the code (Git push) and wait for the Travis CI build and PM2 deployment to complete.

Access Travis CI and the build is successful. Log in to the remote server and enter pm2 list as shown in the following figure:

Visit http://

:8888/user and “Hello, friend” is displayed.

conclusion

This NodeJS application is simple, but involves a lot of knowledge: creating API services, unit testing, ESLint, CI/CD, SSH, Linux operation and maintenance, and requires some practical skills.

Due to the limited space, there are many pits, details too late to speak, if there is a mistake please contact me 📧.

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Continuous integration and deployment of NodeJS applications based on Travis CI + PM2

preface

CI & CD

The preparatory work

The local server, GitHub, and remote (deployment) server can communicate with each other

The local connection making

The local connection remote

Remote connection making

Build the NodeJS application

The use of Travis CI

Adding unit tests

Add ESlint

Use the PM2 Deployment

conclusion

Project source code address

Continuous integration and deployment of NodeJS applications based on Travis CI + PM2

preface

CI & CD

The preparatory work

The local server, GitHub, and remote (deployment) server can communicate with each other

The local connection making

The local connection remote

Remote connection making

Build the NodeJS application

The use of Travis CI

Adding unit tests

Add ESlint

Use the PM2 Deployment

conclusion

Project source code address

Related Posts

Canvas base fuck

Interviewer: Can you write the responsivity principle of Vue2 by hand?

JQuery controls input to only enter numbers and two decimal digits