Gitlab-ci /CD Continuous Integration Test section

I. Gitlab-CI/CD usage scenarios

First, the company uses Gitlab as a workhouse for code release and version control. Gitlab has built-in CI/CD tools, which can be used for code submission and continuous work such as image building, automated testing, automated deployment, etc. :

  • CI: Continuous Integration
  • CD: Continuous Delivery
  • CD: Continuous Deployment

We will only discuss CI continuous integration for the moment. CI is often used to automate tasks that run on a centralized machine, such as packaging of application images, unit testing, deployment, etc., which can save the time spent maintaining correct code during project development iterations.

Automatic test in cases such as CI, in many people in collaborative development process, there may be a different branch of the code push updates frequently, using CI pipe, can be defined in code release at the same time trigger CI unit testing operation, in order to facilitate the discovery of the error in the early development, so as to ensure that all new code submitted does not affect the function of project.

Ii. Understand gitLab-CI /CD workflow

Refer to the following figure to understand the specific workflow and concepts of CI/CD. The yellow part is the main concepts involved, which will be explained in the following article:

  1. Your current repository is on Gitlab and has been configured for the repositorygitlab-runnerService, which is the server that actually performs CI tasks;
  2. Submit code, and the root directory contains a file named.gitlab-ci.ymlA file that is a script used to specify build, test, and deployment processes, as well as CI trigger conditions, similar in concept to the docker-comemess.yml file;
  3. Gitlab detected.gitlab-ci.ymlFile, if the current commit meets the trigger conditions specified in the file, the configuration is usedgitlab-runnerThe service runs the script for testing and so on;
  4. if.gitlab-ci.ymlIf an automated script defined in, it will be judged that the CI fails this time, and the submitter needs to repair the problem code and repeat the submission until the automated CI passes.
  5. Only when there are no problems submitted can the project leader merge into the main branch for subsequent deployment (this article does not cover automatic DEPLOYMENT of CD).

Three, how to write.gitlab-ci.ymlfile

.gitlab-ci. yML file specifies the trigger conditions, work content and workflow of CI. Compiling and understanding this file is the most important step in CI practice. The task content specified in this file is a total of one pipeline, one pipeline contains different execution stages, and each stage contains different specific job script tasks.

.gitlab-ci.yml sample file and common instructions

Pipeline shows that

a.gitlab-ci.ymlThe file triggers to form onepipelineTask flow bygitlab-runnerTo run the processing,pipelineIn thestage,jobThe concepts are as follows, which need to be defined according to the actual situation of the projectstageandjob:

The sample.gitlab-ci.ymlFile running sequence and logic description

Write the YML file in the project root directory as shown in 1.

  1. First, the sequence of stages is defined in the green box. A total of three stages are defined, and they are named build, test and deploy in sequence.

  2. The first blue box defines a job in the Build phase, which is triggered only when a branch of the Tags phase commits, executes the script in the script, packages the project as an image named IMG according to the DockerfileCI file, pushes it to the repository, and then removes the local image.

  3. The second blue box defines a job in the Test phase that has no restricted trigger condition and will be executed on each commit.

    • ImageSpecifies the base image to be used in this phase, which is manually created locally and pushed in advance.
    • servicesWhen the job runs, the container of the specified service version will be initialized and the domain names mongo:27017 and redis:6379 will be exposed respectively. You need to configure the CI configuration file in advance.
    • befor_scriptSpecifies the commands to be executed prior to the execution of the formal script, print Python version information, set the CI-specific test configuration file to the configuration file read by the project, run the test data initialization script, and build the front end of the NPM.
    • scriptSpecifies a formal script execution command to start executing Django unit tests.

  4. Other gitlab-ci.yML files for reference

    • Example: CI/CD docs.gitlab.com/ee/ci/examp…
    • Job Keyword: docs.gitlab.com/ee/ci/yaml/…

Iv. Introduction and use of Gitlab-Runner

Gitlab. yml file (gitlab.yml file), and the execution sequence of each job (gitlab-runner). Gitlab-runner is isolated from the server where gitlab runs. Since building a task often involves compiling, testing, and publishing, this process consumes a lot of system resources. Gitlab-runner can be installed on almost any machine as long as it can communicate with the server where GitLab is located and the Docker warehouse server.

Generally speaking, gitlab-Runner has been configured by the person in charge on Gitlab, and we only need to write. Gitlab. yml file submission code to trigger runner to work. However, for beginners, you may want to execute the job of.gitlab.yml file locally before submitting the code, and then submit the final code after the local debugging succeeds and there is no problem, triggering the CI process of the server. The answer, of course, is yes. The job defined in.gitlab.yml file is actually run by gitlab-runner in a server, so we can install gitlab-runner locally to execute the job in.gitlab.yml manually.

The installation of Gitlab – runner

You can install Gitlab-Runner on almost any machine. Here’s how it works on macs:

  • Brew install gitlab-Runner for MAC

  • Other installation reference: docs.gitlab.com/runner/inst…

  • Gitlab-runner helper install: docker pull gitlab-runner helper

Note: Gitlab-runner needs to rely on the local image of Gitlab-Runner helper. It is best to pull it to the local or private warehouse in advance.

usegitlab-runnerRun the local.gitlab.ymlJob defined in

At present, the local Gitlab-Runner can only execute a specified job at a time and cannot automatically complete the overall flow of the pipeline contained in the YML file, but it is sufficient as a local test. Of course, the Docker service must be started locally and the required base image is available in advance. The job will use the base image to execute the local code in a new directory, which would normally be the/Builds /projects0 directory in the build container for local runners.

  1. Enter the local project root directory, which contains.gitlab.ymlThe directory where the
  2. performgitlab-runnerCommand:

gitlab-runner exec docker test_job --docker-pull-policy=never 3gitlab-runner exec dockerIs the fixed command writing method; –test_jobfor.gitlab.ymlIs defined injob; – --docker-pull-policy=neverIndicates using a local Docker image instead of a network repository

This simple setup and command can run a Gitlab-Runner test locally.

Push code to the GitLab branch

If the local test is ok, you can safely push the code to the remote branch. If the current fork is fork, you need to push the code to upstream to trigger the CI process of the main repository. After the CI passes, the branch code can be merged to the master.

Appendix (Other related operations involved)

Configure the Docker private repository

Since CI process will use some Docker images packaged by ourselves, the upload and download of images will depend on the private warehouse within the group. Here is how to configure the docker warehouse connection:

  • The editordeamon.jsonThe configuration file

Docker Engine – >Docker Engine

Add insecure-registries and DNS configuration as follows:

"insecure-registries": [
    "Your private repository host resolution address"
].
  "dns": [
    "Your private warehouse DNS address"
    ]
Copy the code