Github Action+WebHook implements continuous blog integration deployment to personal servers

Technology blog ssscode.com/pages/33772…

preface

What is the`GitHub Action`,`GitHub Page`

Github Actions help you automate tasks in your software development cycle. They are event-driven and deeply integrated with Github. They can run many Github events, most commonly pushing to the Master branch. But Actions isn’t just about deploying and publishing, they’re containers, and you can literally do anything — the possibilities are endless! You can use them to compress and merge CSS and JavaScript, someone sends you a message when creating an issue in your project repository, and much more… There are no restrictions.
GitHub Page is a free web site that can be used to host a personal static website. It can be used to build a private blog, but also saves the purchase of a server, domain name and a series of complex operations.

Rule: Create a repository named username.github. IO, official document

Use: create an index.html file under the repository to access the url for a quick view

What is the`Coding`

CODING (Coding.net /) is a one-stop software development and management collaboration platform, providing a series of online tools such as Git/SVN code hosting, project collaboration, test management, product library, CI/CD, etc. Help the R&D team quickly adopt agile development and DevOps development methods, improve the efficiency of R&D management, and realize the upgrade of R&D efficiency.
Coding is all about helping developers efficiently develop software in the cloud. Code hosting, project management, presentation platforms, quality management, and more are all designed to help developers accomplish a range of difficult software development activities in the cloud. To provide developers with the ultimate cloud development experience, emphasis on private libraries, emphasis on team collaboration, emphasis on integrated experience, emphasis on speed of access.
Why is Coding used?

The role of Coding is similar to that of Gitee, which is deployed in China and has a significant speed increase, and can also be included in Baidu. For well-known reasons, it is very impressive to visit GitHub in China. I also tried to directly obtain the code of GitHub at the beginning, but the speed was too slow and the efficiency was very low. Therefore, I finally chose Coding as the warehouse mirror.

What is the`Webhook`

Webhook is an API concept, the term “network hook”, sometimes referred to as “reverse API”. Because it provides API rules, you need to design the API to use. Webhook will make HTTP requests to your application, typically POST requests, and the application is request-driven. The more things we can describe with events, the greater the scope of Webhook.
To be precise, Webhook is a Web callback or HTTP push API, a way to provide real-time information to apps or other applications. Webhook sends data as soon as it is generated, meaning you can receive it in real time. With WebHooks, you can get push notifications when certain events occur on the server. You can “subscribe” to activities using Webhooks.

Requirements and Scenarios

As we all know, GitHub as the world’s largest gay dating community, in order to communicate with you more convenient and friendly positive 🐶, my source code storage address is stored in GitHub, the content includes blog updates, some test cases, tutorials, Demo and so on.

Because my blog is deployed on the personal server of Ali Cloud, sometimes the blog is updated frequently. At the beginning, I linked to the server through SSH and then manually updated it by commands such as git clone, git pull and NPM run build. There are several problems:

Frequent login to the server is tedious
GitHubSpeed problem, each update has to wait a long time
Server performance issues for low configuration server executionnpm run buildPacking is difficult

Ideas and Flow chart

Train of thought

Local computer throughgitSubmitted to theGitHubwarehouse
GitHub ActionListen to thepush eventEvents triggerci.ymlExecute the script
buildPackage generated deployment files pushed toGitHub gh-pagesBranch andCoding masterbranch
CodingSet up thewebhookListening to thepush eventEvents triggerwebhookhook
webhookThe hook communicates to the personal server within the startuphttp server, verify identity and warehouse, executewebhook.shThe script
webhook.shThe scriptcdEnter thenginxUnder the blog deployment directory, proceedgit pullThe update operation
Generate logs based on the results, or add mail notification capabilities (combinedNodemailerLibrary implementation)

The flow chart

Create a personal Access Token

Next, automated deployment and continuous integration will use HTTPS to submit code to the repository, where the access Token personal access token is configured as an environment variable to be used by actions and scripts.

GitHub Token

The first step is to generate a Github token according to the official documentation.

Second, store the key in Settings/Secrets of the current repository.

Settings/Secrets is where private environment variables are stored. The name of the environment variable can be arbitrary, in this case ACCESS_TOKEN. If you do not use this name, change the variable names in the.github/workflows/ci.yml script.

Coding Token

The first step is to generate a coding token according to the official documentation.

Step 2: Same as GitHub

Several kinds of schemes

:::tip Here are examples of several schemes I have tried. The advantages and disadvantages of each scheme will be carefully analyzed and introduced, as well as the methods I gradually improved and tried, and the final implementation. : : :

The SCP command is deployed directly to the server

The main idea

Copy the local deployment file directly to the remote server by using the SCP command

SCP is short for Secure Copy. It is a secure remote file copy command based on SSH login in Linux. SCP transfer is encrypted.

The NPM run build package is compiled locally to obtain the deployment file

Update directly by uploading replacement source files

# 1. This command will iterate the dist file in the current directory and upload it to the blog directory: /blog/dist
scp -r ./dist root@ip/usr/local/app/blog/
# 2. Enter the server password and wait for the transfer to complete
# Of course, you can also upload without password through SSH, but this does not achieve my ultimate goal, I will not repeat here
Copy the code

The final result

Performance issues with server compilation have been resolved
GitHub was slow to pull updates
For each update, the steps such as package compilation and login operation are still tedious

GitHub Action Pages

The main idea

Gh-pages continuous integration and deployment is realized by using the Page and Action services provided by GitHub

Gh-pages build a lot of tutorials online, I will not repeat the specific implementation. Here I offer one of my solutions:

Multi-address deployment is realized by dynamically generating gh-Pages branches and pushing them to github repository

As my blog making address: JS – banana. Making. IO/vuepress

scripting

Github >workflows>ci.yml was created in the root directory

Ci. Yml file

jobs: # workflow
  build: # custom name
  runs-on: ubuntu-latest # Run in the VM environment ubuntu- Latest

  strategy:
    matrix:
    node-version: [14.x]

  steps: Step #
    - name: Checkout Step # 1
    uses: actions/checkout@v1 # Use the action. The format is userName/repoName.
    Check out the repository and get the source code. The official actions library: https://github.com/actions
    - name: Use Node.js The ${{ matrix.node-version }} Step # 2
    uses: actions/setup-node@v1 # install nodejs
    with:
      node-version: The ${{ matrix.node-version }} # version
    - name: Deploy # Step 3 Deploy to Github GH-Pages
    env: Set environment variables
      GITHUB_TOKEN: The ${{ secrets.ACCESS_TOKEN }} # token private variable
    run: | npm install npm run build cd ./dist git init git config user.name "name" git config user.email "email" git add . git commit -m "$(date) Update from Action" git push --force --quiet "https://JS-banana:${GITHUB_TOKEN}@github.com/JS-banana/vuepress.git" master:gh-pagesCopy the code

The page item can be used as an alternate address, directed to our own DNS server via CNAME. CNAME: alias record. This recording allows you to map multiple names to another domain name.

Run the echo ‘ssscode.com’ > CNAME command to generate the CNAME file, and then place the CNAME file in the generated dist directory, which can be handled by the bash script

Let’s adjust the above script again

jobs: # workflow
  build: # custom name
  runs-on: ubuntu-latest # Run in the VM environment ubuntu- Latest

  strategy:
    matrix:
    node-version: [14.x]

  steps: Step #
    - name: Checkout Step # 1
    uses: actions/checkout@v1 # Use the action. The format is userName/repoName.
    Check out the repository and get the source code. The official actions library: https://github.com/actions
    - name: Use Node.js The ${{ matrix.node-version }} Step # 2
    uses: actions/setup-node@v1 # install nodejs
    with:
      node-version: The ${{ matrix.node-version }} # version
    - name: Deploy # Step 3 Deploy to Github GH-Pages
    env: Set environment variables
      GITHUB_TOKEN: The ${{ secrets.ACCESS_TOKEN }} # token private variable
    run: npm install && npm run deploy
Copy the code

Sh file and add the “deploy”: “bash deploy.sh” command to the package.json field scripts

#! /usr/bin/env sh

Make sure the script throws any errors it encounters
set -e

# date
nowDate=$(date "+%Y-%m-%d %H:%M:%S")

Generate static files
npm run build

Go to the generated folder
cd ./dist

# CNAME
echo 'www.ssscode.com\ssscode.com' > CNAME  # Custom domain name

# github url
githubUrl=https://JS-banana:${GITHUB_TOKEN}@github.com/JS-banana/vuepress.git

Git user info
git config --global user.name "JS-banana"
git config --global user.email "[email protected]"

# commit
git init
git add -A
git commit -m "The deploy. Sh = = = > update:${nowDate}"
git push -f $githubUrl master:gh-pages # Push to Github

cd - Return to the directory where you started
rm -rf ./dist
Copy the code

At this point, we have implemented continuous integration through GitHub Action, packaged the deployment file and pushed it to the GH-Pages branch.

Coding is deployed synchronously with GitHub

In fact, the principle of this step is the same as the above GitHub Action Pages, and the most important step we need to do is to configure the Coding token to GitHub repository Settings/Secrets. Add an environment variable CODING_TOKEN. This method also applies to Gitee. If you want to use Gitee, you can try it yourself.

The ci.yml file is added

Env: ${{secrets.ACCESS_TOKEN}} # github token private variable+ CODING_TOKEN: ${{secrets.CODING_TOKEN}} # coding token private variable
Copy the code

Change the deploy.sh file to

  # github url
  githubUrl=https://JS-banana:${GITHUB_TOKEN}@github.com/JS-banana/vuepress.git

+ # coding url
+ # attention!! Here you need to use the user name and token of the personal token provided by coding (https://[name]:[token]@e.coding.net/xx)
+ codingUrl=https://ptzv1yuleer1:${CODING_TOKEN}@e.coding.net/ssscode/blog/vuepress.git Git push -f $githubUrl master:gh-pages # push to github- CD - # Return to the directory where you started
- rm -rf ./dist
  
+ git push -f $codingUrl master

+ CD - # Return to the directory where you started
+ rm -rf ./dist
Copy the code

Webhook and Coding implementations are continuously deployed

Implementation approach

First, implement a nodeJS HTTP server for receiving requests, namely the Webhook. js hook service. Here I suggest that the domain name resolves a secondary domain name for webhook exclusive use, such as: Webhook.ssscode.com, of course, directly under the current project with the /webhook path as the trigger route is also possible. Here we take the first scenario as an example, using nodeJS ‘pM2 daemon to start webhook.js, and then using nginx reverse proxy to map locally started services to webhook.ssscode.com.

Create webhook. Js

The main use of NodeJs module HTTP (create server) and child_process (execute bash script)

Without further ado, let’s get right to the code

// webhook.js
const server = require('http');
const { spawn } = require('child_process');

server
  .createServer((req, res) = > {
    // accept request
    console.log(` accept the request:The ${new Date()}`);
    // Receive the POST request
    if (req.method === 'POST') {
      //TODO:Secret authentication

      let data = ' ';

      req.on('data'.chunk= > {
        data += chunk;
      });

      req.on('end'.() = > {
        // console.log(JSON.parse(data));
        try {
          const reqData = JSON.parse(data);
          // Determine the identity
          if(reqData.pusher.username ! = ='xxx') { // Coding Personal Access Token User name
            res.writeHead(400);
            res.end('noooo! ');
            return;
          }
          // Determine the branch master
          if (reqData.ref === 'refs/heads/master') {
            // Determine the warehouse
            const repository_name = reqData.repository.name;
            runCommand('sh'[`${repository_name}.sh`].console.log);
          }
          // response
          res.writeHead(200);
          res.end('ok');
        } catch (error) {
          console.log('error：', error);
          res.writeHead(500);
          res.end('error! '); }}); }else {
      res.writeHead(404);
      res.end('no info! ');
    }
  })
  .listen(3010); / / port

// run command
function runCommand(cmd, args, callback) {
  let response = ' ';
  const child = spawn(cmd, args);
  child.stdout.on('data'.buffer= > {
    response += buffer.toString();
  });
  child.stdout.on('end'.() = > callback(response));
}
Copy the code

The core code is runCommand function, server service receives the request parameters to verify identity and warehouse information, meet the conditions to execute the corresponding script. There is no processing validation of the key as in the Github-Webhook-Handler package, just simple authentication and conditions. (Github, coding, etc., can set authentication key when configuring Webhook)

Next we write our bash script, which can then be optimized to execute the corresponding script under the corresponding project (that is, if there are multiple projects or blogs, the relevant script is created under the corresponding project and executed by bash).

Creating a bash script

Vuepress. sh Core code

#! /usr/bin/env sh
Make sure the script throws any errors it encounters
set -e

cd /usr/local/app/vuepress-blog/dist

echo 'start===>git'

# override update
git fetch --all
git reset --hard origin/master
# git clean -f
# git pull

cd - Return to the directory where you started
Copy the code

In order to facilitate recording and viewing and expand notification messages, exception judgment processing and log output are added here

- CD - # Return to the directory where you started

+ function log_info (){
+ DATE_N=`date "+%Y-%m-%d %H:%M:%S"`
+ USER_N=`whoami`
+ echo "${DATE_N} ${USER_N} the execute $0 [INFO] $@" > > / usr/local/app/webhook/logInfo. TXT # execution log print path to success
+}

+ function log_error (){
+ DATE_N=`date "+%Y-%m-%d %H:%M:%S"`
+ USER_N=`whoami`
+ echo -e "\033[41;37m ${DATE_N} ${USER_N} execute $0 [ERROR] $@ \033[0m" >> /usr/local/app/webhook/logError.txt # Path to print execution failure logs
+}

+ if [ $? -eq 0 ]
+ then
+ log_info "$@ sucessed."
+ echo -e "\033[32m $@ sucessed. \033[0m"
+ else
+ log_error "$@ failed."
+ echo -e "\033[41;37m $@ failed. \033[0m"
+ exit 1
+ fi

+ trap 'fn_log "DO NOT SEND CTR + C WHEN EXECUTE SCRIPT !!!! "' 2

+ CD - # Return to the directory where you started
Copy the code

Create the nginx configuration

Nginx configuration

server {
  # the network
  listen 80;
  server_name webhook.ssscode.com; # the domain name

  # Listen on local services
  location / {
    # Enable reverse proxy
    proxy_passhttp://127.0.0.1:3010/; }}Copy the code

To start the test, execute Node./webhook.js

pm2

PM2 is a node process management tool. It can be used to simplify many node application management tasks, such as performance monitoring, automatic restart, load balancing, and so on. PM2 Simple user manual

To facilitate NodeJs management and monitoring, it is recommended to start the service using pm2 start webhook.js

Github or Gitee with Webhook

Because I adopt the method of Coding combined with Webhook, if you are interested in other ways, you can also build it by yourself. The principle is similar. At present, Github and Gitee also have open source NPM package to provide fast building of Webhook communication service. Lot – webhook handler, gitee webhook – handler

conclusion

Done ~

In general, I tried many methods repeatedly, groped for ideas, and made continuous improvement. Finally, I realized my original idea, but I did take many detours. Here, the whole process recorded for their own reference and in order to deepen understanding, I hope to help partners in need, less detours ~

If you configure all kinds of services and code business logic, it does take time and energy, the technical points involved are also relatively miscellaneous, many places can only achieve barely use, is far from, the whole process is to learn and try. However, it is also very helpful for my technical growth and business understanding

If you’re interested in Hexo, you can try it out. I’m working on it, but it’s not working as well as I’d like. Let’s see if there’s anything to write about later

Later, it is also necessary to learn more about Docker

reference

Coding.net/ docs.github.com/cn/actions xugaoyi.com/pages/6b9d3… Docs.github.com/cn/github/a… Help.coding.net/docs/member… Ipcmen.com/ www.ruanyifeng.com/blog/2016/0… Juejin. Cn/post / 684490…

Github Action+WebHook implements continuous blog integration deployment to personal servers

preface

What is theGitHub Action,GitHub Page

What is theCoding

What is theWebhook

Requirements and Scenarios

Ideas and Flow chart

Train of thought

The flow chart

Create a personal Access Token

GitHub Token

Coding Token

Several kinds of schemes

The SCP command is deployed directly to the server

The main idea

The final result

GitHub Action Pages

The main idea

scripting

Coding is deployed synchronously with GitHub

Webhook and Coding implementations are continuously deployed

Implementation approach

Create webhook. Js

Creating a bash script

Create the nginx configuration

pm2

Github or Gitee with Webhook

conclusion

reference

Related Posts

Vue small note — grocery store

NPM third party module FS module

Javascript event loop

What is the`GitHub Action`,`GitHub Page`

What is the`Coding`

What is the`Webhook`