Vue + Node.js implements automatic deployment tools from 0 to 1

A recently written NPM package for automated deployment, zuo-deploy, executes server deployment scripts with the click of a button to complete functional update iterations. The client uses Vue + ElementUI, koA + Socket + KOA-session, etc. Basic function code is less than 300 lines, has been open source on Github. Welcome to Star and Fork. Here is the specific implementation details, ideas.

The directory structure

├ ─ ─ binCommand line tool command│ ├ ─ ─ start. Js# zuodeploy start Execution entry│ └ ─ ─ zuodeploy. Js# zuodeploy command entry, configured in the bin property of package.json├ ─ ─ docImages# readme.md Document image├ ─ ─ frontend# koA -static server directory│ └ ─ ─ index. HTML# Vue + ElementUI + axios + socket.io├ ─ ─ server# the service side│ ├─ ├─ Logg.js │ ├─ Utills │ ├─ Logg.js# log4js │ │ └ ─ ─ runCmd. Js# node child_process spawn (execute shell script, pM2 service start)│ └ ─ ─ index. JsMain service (KOA interface, static service + socket + execute shell script)├ ─ ─. Eslintrc. CJS# esLint configuration file + prettier├ ─ ─ the args. Json# For pM2 after modification, cross-file transfer port, password parameters├ ─ ─ CHANGELOG. The md# release feature iteration record├ ─ ─ the deploy - master. ShFor testing, service pairs are enabled in the current directory. Click the Deploy button to execute the script├ ─ ─ index. Js# zuodeploy start execution file used to execute the pm2 start server/index.js master service├ ─ ─ package. Json# project description file, NPM package name, version number, CLI command name,├ ─ ─ the publish. Sh# NPM publish publish script└ ─ ─ the README. Md# Use documentation
Copy the code

Front and back end technology stacks, dependencies

• Front-end/client
  • Static HTML + CSS, non-front-end engineering, libraries are introduced in the form of CDN, through the library in UMD package exposed global variables
  • Vue3, MVVM framework, does not manipulate dom
  • Element-plus, unified and beautified basic form style
  • Axios, request interface
  • Socket. IO to receive real-time deployment logs
• The service side
  • Based on Node.js technology stack, no database
  • Commander, used to generate the zuodeploy command runtime help files, prompts, and zuodeploy start execution entry
  • Prompts, refer to vue-create, to guide the user to a port and password
  • Koa, HTTP server, provides interface, static service run container (similar to Nginx, Tomcat, etc.)
  • Koa-bodyparser for PARSING POST request parameters (required for login authentication interface)
  • Koa-router, used to execute different methods on different interfaces (paths such as /login, /deploy, etc.)
  • Koa-session: used for interface authentication to prevent others from requesting deployment after obtaining the deployment interface
  • Koa -static, like nginx starts a static service
  • Socket. IO, the socket server, sends logs to the front end in real time when git pull and NPM run build are deployed for a long time
    • Common interfaces may need to be fully deployed before the results are available
  • Log4js, time-stamped log output
  • Pm2 is directly executed. When terminal terminates, the service will be shut down, and pM2 is used for silent execution in the background

Basic function realization idea

Initial goal: Click the Deploy button on the front end page to have the server perform the deployment directly and return the deployment log to the front end

How do you do that?

• 1. There should be a front page with deployment buttons and log display area.
• 2. The front-end page must have a server to interact with the server
  • 2.1 Provide an interface. Click deploy on the front page to request the interface and know when to deploy.
  • 2.2 How can A Back-end Interface Perform a Deployment Task After Receiving a Request?
  • 2.3 How to collect and send logs executed by shell scripts to the front-end? Spawn also supports log output

Technology stack determination:

• Vue + ElementUI basic page layout + basic logic, AXIos requests interface data
• 2. Use the Node technology stack to provide the server
  • 2.1 Implement the interface using KOA/KOA-router
  • 2.2 Generally, shell scripts are executed. Node uses the built-in spawn process to execute shell script files and run commands in terminal
  • 2.3 During spawn execution, the child process stdout and stderr can obtain the script execution log and return it to the front end

In consideration of front-end page deployment, you can use koA-static to enable the static file service together with the KOA Server service to support front-end page access

Instead of using front-end engineering @vue/ CLI, we directly use static HTML and introduce VUE through CDN

1. Client Vue+ElementUI+ Axios

The frontend service we put frontend/index.html, koa-static static service points directly to the frontend directory to access the page

The core code is as follows:

Note: CDN links are // relative path, need to use HTTP service to open the page, can not be opened in the form of ordinary File! You can wait until the koA is finished and the service is started

<head>
  <title>zuo-deploy</title>
  <! -- Import style -->
  <link rel="stylesheet" href="//unpkg.com/element-plus/dist/index.css" />
  <! Vue 3 -->
  <script src="//unpkg.com/vue@next"></script>
  <! Import component library -->
  <script src="//unpkg.com/element-plus"></script>
  <script src="https://unpkg.com/axios/dist/axios.min.js"></script>
</head>

<body>
  <div id="app" style="margin:0 20px;">
    <el-button type="primary" @click="deploy">The deployment of</el-button>
    <div>
      <p>Deployment logs:</p>
      <div class="text-log-wrap">
        <pre>{{ deployLog }}</pre>
      </div>
    </div>
  </div>
  <script>
    const app = {
      data() {
        return {
          deployLog: 'Click the button to deploy',}},methods: {
        deploy() {
          this.deployLog = 'Back-end deployment, please wait... '
          axios.post('/deploy')
            .then((res) = > {
              // When deployment is complete, return log
              console.log(res.data);
              this.deployLog = res.data.msg
            })
            .catch(function (err) {
              console.log(err);
            })
        }
      }
    }

    Vue.createApp(app).use(ElementPlus).mount('#app')
  </script>
</body>
Copy the code

2. The server koa+koa-router+koa-static

The KOA starts the HTTP server and writes the deploy interface for processing. Koa-static Enables the static service

// server/index.js
const Koa = require("koa");
const KoaStatic = require("koa-static");
const KoaRouter = require("koa-router");
const path = require("path");

const app = new Koa();
const router = new KoaRouter();

router.post("/deploy".async (ctx) => {
  // Execute the deployment script
  let execFunc = () = > {};
  try {
    let res =  await execFunc();
    ctx.body = {
      code: 0.msg: res,
    };
  } catch (e) {
    ctx.body = {
      code: -1.msg: e.message, }; }}); app.use(new KoaStatic(path.resolve(__dirname, ".. /frontend")));
app.use(router.routes()).use(router.allowedMethods());
app.listen(7777.() = > console.log('Service monitoring'The ${7777}Port `));
Copy the code

Run the project

1. In the current project directory, executenpm initInitialization package. Json
2. npm install koa koa-router koa-static --saveInstalling dependency packages
3. node server/index.jsRun the project. If port 7777 is occupied, change it to another port

Visit http:// 127.0.0.1:7777 to access the page and click Deploy to request success

3.Node executes shell scripts and outputs logs to the front-end

Spawn of the built-in node module child_process runs the terminal command, including the sh script file of the shell script

Take a look at a demo and create a test directory for testExecShell

// testExecShell/runCmd.js
const { spawn } = require('child_process');
const ls = spawn('ls'['-lh'.'/usr']); // Run ls -lh /usr

ls.stdout.on('data'.(data) = > {
  // Ls generates the terminal log in the console
  console.log(`stdout: ${data}`);
});

ls.stderr.on('data'.(data) = > {
  // If an error occurs, the error is output from here
  console.error(`stderr: ${data}`);
});

ls.on('close'.(code) = > {
  // Exit after execution is 0
  console.log(`child process exited with code ${code}`);
});
Copy the code

Run node testExecShell/ runcmd.js to execute ls -lh /usr with Node and receive log information via ls.stdout and print it

To get back to the point where a shell script is required, replace ls -lh /usr with the sh script file. Sh. Let’s try it out

// testExecShell/runShell.js
const { spawn } = require('child_process');
const child = spawn('sh'['testExecShell/deploy.sh']); // Run the sh deploy.sh command

child.stdout.on('data'.(data) = > {
  // Shell logs are collected here and can be returned to the front end via the interface
  console.log(`stdout: ${data}`);
});

child.stderr.on('data'.(data) = > {
  // If an error occurs, the error is output from here
  console.error(`stderr: ${data}`);
});

child.on('close'.(code) = > {
  // Exit after execution is 0
  console.log(`child process exited with code ${code}`);
});
Copy the code

To create a shell script for execution, run sh estExecShell/deploy.sh to check whether the script is executable. If the script is not executable, add (chmod +x filename).

# /testExecShell/deploy.sh
echo 'execution PWD'
pwd
echo 'Execute git pull'
git pull
Copy the code

Running node testExecShell/runShell js can make node deploy. Sh scripts. The following figure

Reference: child_process-node. js Built-in module notes

4. The deploy interface integrates the shell script function

Modify the deploy interface, add a runCmd method, and execute the deploy. Sh deployment script in the current directory. After the deployment is complete, the interface sends a log response to the front end

// Create a new server/ indexexecshell. js, copy the server/index.js content in, and make the following changes
const rumCmd = () = > {
  return new Promise((resolve, reject) = > {
    const { spawn } = require('child_process');
    const child = spawn('sh'['deploy.sh']); // Run the sh deploy.sh command

    let msg = ' '
    child.stdout.on('data'.(data) = > {
      // Shell logs are collected here and can be returned to the front end via the interface
      console.log(`stdout: ${data}`);
      // The common interface can only return logs once. You need to collect all logs once and return them to the front-end at end
      msg += `${data}`
    });

    child.stdout.on('end'.(data) = > {
      resolve(msg) // Interface resolve is returned to the front end
    });

    child.stderr.on('data'.(data) = > {
      // If an error occurs, the error is output from here
      console.error(`stderr: ${data}`);
      msg += `${data}`
    });

    child.on('close'.(code) = > {
      // Exit after execution is 0
      console.log(`child process exited with code ${code}`);
    });
  })
}

router.post("/deploy".async (ctx) => {
  try {
    let res =  await rumCmd(); // Execute the deployment script
    ctx.body = {
      code: 0.msg: res,
    };
  } catch (e) {
    ctx.body = {
      code: -1.msg: e.message, }; }});Copy the code

After the modification is complete, run node Server/IndexExecshell. js to start the latest service and click Deploy. The interface runs normally, as shown in the following figure

Sh of the current directory is executed without the corresponding file. Place testExeclShell/deploy.sh in the current directory and click Deploy

This completes the automated deployment infrastructure.

Function optimization

1. Output logs using sockets in real time

In the example above, the common interface responds to the front-end after the deployment script is executed. If the script contains time-consuming commands such as Git pull and NPM run build, the front-end page does not have log information, as shown in the following figure

Test the shell

echo 'execution PWD'
pwd
echo 'Execute git pull'
git pull
git clone [email protected]:zuoxiaobai/zuo11.com.git The command takes a long time
echo 'Deployment complete'
Copy the code

Here we modify it to use socket. IO to send deployment logs to the front end in real time

Socket. IO is divided into two parts: client and server

Client code

<! -- frontend/indexSocket.html -->
<script src="https://cdn.socket.io/4.4.1/socket.io.min.js"></script>
<script>
  // Vue Mounted hook connects to the socket server
  mounted() {
    this.socket = io() // Connect to the socket server, send an HTTP request, and then forward the 101 WS protocol
    // Subscribe to the deployment log, get the log, push it into the array and display it to the front end
    this.socket.on('deploy-log'.(msg) = > {
      console.log(msg)
      this.msgList.push(msg)
    })
  },  
</script>
Copy the code

The socket. IO code is introduced in the back-end KOA

// server/indexSoket.js
// npm install socket.io --save
const app = new Koa();
const router = new KoaRouter();

// Enable the socket service
let socketList = [];
const server = require("http").Server(app.callback());
const socketIo = require("socket.io")(server);
socketIo.on("connection".(socket) = > {
  socketList.push(socket);
  console.log("a user connected"); // The front-end call IO (), then the connection is successful
});
// The returned socketIo object can be used to broadcast messages to the front-end

runCmd() {
  // Some core code
  let msg = ' '
  child.stdout.on('data'.(data) = > {
    // Shell logs are collected here and can be returned to the front end via the interface
    console.log(`stdout: ${data}`);
    socketIo.emit('deploy-log'.`${data}`) // The socket sends messages to the front-end in real time
    // The common interface can only return logs once. You need to collect all logs once and return them to the front-end at end
    msg += `${data}`
  });
  // ...
  child.stderr.on('data'.(data) = > {
    // If an error occurs, the error is output from here
    console.error(`stderr: ${data}`);
    socketIo.emit('deploy-log'.`${data}`) // The socket sends messages to the front-end in real time
    msg += `${data}`
  });
}
// app.listen needs to be changed to the server object with socket service above
server.listen(7777.() = > console.log('Service monitoring'The ${7777}Port `));
Copy the code

We add the above code in the demo before, can complete the socket transformation, the node server/indexSocket js, open 127.0.0.1:7777 / indexSocket. HTML, click on deployment, can see the following result. The demo access address is complete

Issues related to

1. As for the HTTP to WS protocol, we can open the F12 NetWork panel to see the front-end socket connection steps

• GET http://127.0.0.1:7777/socket.io/?EIO=4&transport=polling&t=Nz5mBZkGet the sid
• POST http://127.0.0.1:7777/socket.io/?EIO=4&transport=polling&t=Nz5mBaY&sid=DKQAS0fxzXUutg0wAAAG
• GET http://127.0.0.1:7777/socket.io/?EIO=4&transport=polling&t=Nz5mBav&sid=DKQAS0fxzXUutg0wAAAG
• Ws: / / 127.0.0.1:7777 / socket. IO /? EIO=4&transport=websocket&sid=DKQAS0fxzXUutg0wAAAG

You can see socket data in ws

2. The HTTP request success Status Code is typically 200, and the WS Status Code is 101 Switching Protocols

2. Add authentication for the deployment interface

The above functions are only implemented by the interface without permission control. Anyone who knows the address of the interface can request the interface through Postman to trigger deployment. The following figure

For the sake of security, we add authentication for the interface and add a password login function at the front end. In this case, koA-session is used for authentication. The request can be successful only when the login state is used

// server/indexAuth.js
// npm install koa-session koa-bodyparser --save
// ..
const session = require("koa-session");
const bodyParser = require("koa-bodyparser"); // Post request parameter parsing
const app = new Koa();
const router = new KoaRouter();

app.use(bodyParser()); // Process the POST request parameters

/ / integration session
app.keys = ['Custom security string']; // 'some secret hurr'
const CONFIG = {
  key: "koa:sess" /** (string) cookie key (default is koa:sess) */./** (number || 'session') maxAge in ms (default is 1 days) */
  /** 'session' will result in a cookie that expires when session/browser is closed */
  /** Warning: If a session cookie is stolen, this cookie will never expire */
  maxAge: 0.5 * 3600 * 1000./ / 0.5 h
  overwrite: true /** (boolean) can overwrite or not (default true) */.httpOnly: true /** (boolean) httpOnly or not (default true) */.signed: true /** (boolean) signed or not (default true) */.rolling: false /** (boolean) Force a session identifier cookie to be set on every response. The expiration is reset to the original maxAge, resetting the expiration countdown. (default is false) */.renew: false /** (boolean) renew session when session is nearly expired, so we can always keep user logged in. (default is false)*/}; app.use(session(CONFIG, app)); router.post("/login".async (ctx) => {
  let code = 0;
  let msg = "Login successful";
  let { password } = ctx.request.body;
  if (password === 888888 ` `) { // 888888 is the password
    ctx.session.isLogin = true;
  } else {
    code = -1;
    msg = "Password error";
  }
  ctx.body = {
    code,
    msg,
  };
});

router.post("/deploy".async (ctx) => {
  if(! ctx.session.isLogin) { ctx.body = {code: -2.msg: "Not logged in"};return;
  }
  // There is a login state, and the deployment is performed
})
Copy the code

Front-end related changes, add a password input box, a login button

<! -- frontend/ indexauth.html Note id="app" package -->
<div class="login-area">
  <div v-if=! "" isLogin">
    <el-input v-model="password" type="password" style="width: 200px;"></el-input>
    &nbsp;
    <el-button type="primary" @click="login">The login</el-button>
  </div>
  <div v-else>Is logged in</div>
</div>
<script>
data() {
  return {
    isLogin: false.password: ' '}},methods: {
  login() {
    if (!this.password) {
      this.$message.warning('Please enter your password')
      return
    }
    axios.post('/login', { password: this.password })
      .then((response) = > {
        console.log(response.data);
        let { code, msg } = response.data
        if (code === 0) {
          this.isLogin = true
        } else {
          this.$message.error(msg)
        }
      })
      .catch(function (err) {
        console.log(err);
        this.$message.error(err.message)
      })
  }
}
</script>
Copy the code

Node server/indexAuth js, open 127.0.0.1:7777 / indexAuth. HTML, and can be deployed after successful login

3. Encapsulate the CLI tool into an NPM package

Why encapsulate as NPM package, use command line tool to start service. It is mainly simple and easy to use. If you do not use the command line tool form, there are three steps:

1. Start by downloading the code to the server
2. npm install
3. Js or pm2 start index.js -n XXX Starts the service

Switching to the NPM package command line tool format takes only the following two steps and is much more time saving

1. npm install zuo-deploy pm2 -g
2. Running Zuodeploy Start automatically starts the service using PM2

Let’s start with a simple example of the steps to create an NPM package and upload it to the official NPM library

• NPM account is required. If not, you can register one at www.npmjs.com/. My username is ‘guoqzuo’.
• Create a folder to store the NPM package contents, such as npmPackage
• NPM init initials. json (‘zuoxiaobai-test’)
  • There are two forms of package name: normal package vue-cli and scope package @vue/cli. For the difference, see @vue/cli.
• The default entry is index.js, which exposes one variable and one method

// index.js
module.exports = {
  name: 'Write an NPM package'.doSomething() {
    console.log('This NPM exposes a method')}}Copy the code

• To do this, create a publish script and execute (Linux chmod +x publish.sh; . / publish. Sh)

# publish.sh
npm config set registry=https://registry.npmjs.org
npm login If you have OTP, the mailbox will receive the verification code, enter it
After a successful login, the status will be saved for a short time. You can directly NPM pubish
npm publish [root@xxx/XXX] [root@xxx]
npm config set registry=https://registry.npm.taobao.org # Restore Taobao mirror
Copy the code

Go to npmjs.org and search for the corresponding package

Using this NPM package, create testNpm/index.js

const packageInfo = require('zuoxiaobai-test')

console.log(packageInfo) 
packageInfo.doSomething()
Copy the code

NPM init: install zuoxiaobai-test –save; Then the node index.js is executed as shown in the following figure. The NPM package is called normally

So we know how to write an NPM package and upload it to the official NPM library.

Now, let’s look at how to integrate CLI commands into the NPM package. For example: after NPM install@vue /cli -g, a vue command is added to the environment variable. Use vue Create XX to initialize a project. This is usually the CLI tool.

There is typically a bin property in package.json that creates custom commands for the NPM package

// package.json
"bin": {
    "zuodeploy": "./bin/zuodeploy.js"
  },
Copy the code

After NPM install xx -g is installed globally, the zuodeploy command is generated. When the zuodeploy command is executed, the bin/zuodeploy.js command is executed

For local development, after configuration, run sudo NPM link in the current directory to link the zuodeploy command to a local environment variable. Running zuodeploy in any terminal will execute the file in the current project. NPM unlink can be used to unlink

The CLI uses commander to generate help documents and manage command logic. The code is as follows

// bin/zuodeploy.js
#!/usr/bin/env node

const { program } = require("commander");
const prompts = require("prompts");

program.version(require(".. /package.json").version);

program
  .command("start")
  .description("Enable deployment listening service") // description + action prevents command splicing files from being found
  .action(async() = > {const args = await prompts([
      {
        type: "number".name: "port".initial: 7777.message: "Please specify deployment service listening port:".validate: (value) = >value ! = ="" && (value < 3000 || value > 10000)?The port number must be between 3000 and 10000
            : true}, {type: "password".name: "password".initial: "888888".message: "Please set login password (default: 888888)".validate: (value) = > (value.length < 6 ? The password needs more than 6 digits : true),},]);require("./start")(args); // args is {port: 7777, password: '888888'}
  });

program.parse();
Copy the code

Commander allows you to quickly manage, generate help documents, and assign execution logic to specific instructions

Prompts prompts. In the code above, where the start command is specified, Zuodeploy Start will take a prompt to retrieve the parameter, and then execute bin/start.js

In start.js, we can copy all the server/index.js code to zuodeploy start and click deploy

4. Stability improvement – PM2 transformation

In order to improve the stability, we can execute pm2 SRC /index.js in start.js code so that the service is more stable and reliable. In addition, we can add log4js to output the log with timestamp, which is conducive to troubleshooting problems.

• Code reference: zuo- deploy-github
• All test demo addresses: zuo-deploy implement demo-fedemo-github

The last

This is the implementation of zuo-deploy. The code is written at will. Welcome star, fork and improve PR!

Other problems

Why is there only one HTML in the front-end/client that is not engineered

1. Organizing code in a front-end engineering way is heavy and unnecessary
2. Here the function is relatively simple, only deployment button, deployment log view area, authentication (enter password) area
3. Easy to deploy, directly koA-static enable static services can be accessed, no need to package build

Why change from Type: Module to plain CommonJS

Json sets type: module to ES Modules by default. Some node methods have some problems

This can be done by changing the file suffix to.cjs, but it is better to remove type: module and use the node default package

1. __dirnameAn error.__dirnameThis is important for CLI projects. __dirname is used when you need to use files in the current project rather than the directory in which zuodeploy Start was executed
2. require(“.. /package.json”) to import xx from ‘.. /package.json’ error importing JSON file