Jenkins and Kubernetes continue to integrate Springboot + Helm

Environment introduction:

• Jenkins： 2.172
• Tokens Helm: 2.13.1
• Kubernetes： 1.14.0
• Chart Repo warehouse address: chart.mydlq.club
• Github address of test project: Github.com/my-dlq/spri…
• Helm chart template Github address: Github.com/my-dlq/spri…

Install Jenkins in Kubernetes

For details, see Installing Jenkins in Kubernetes.

2. Jenkins install plugins

To facilitate integration with Maven, Kubernetes, configuration files, etc., there are several additional plug-ins that need to be installed here. Plug-ins can be installed in System Administration – > Plug-ins Management – > Optional Plug-ins as listed below.

• (1) the Git plug-in
• (2) the Docker plug-in
• (3) Kubernetes
• (4) Kubernetes Cli
• (5) the Config File Provider
• 6 Pipeline Utility Steps

1. Git plug-in

Git plug-ins are installed by default with the Jenkins installation, so a separate installation is not required. Git tools can be used to download the source code github, Gitlab, etc.

Docker plugin

The Docker plugin is installed by default in the Jenkins installation, so you don’t need to install it separately. Docker plug-in can be used to set up the Docker environment, run Docker commands, configure remote Docker warehouse credentials, etc.

3, Kubernetes

The purpose of the Kubernetes plugin is to be able to dynamically configure the Jenkins agent using the Kubernetes cluster (using the Kubernetes scheduling mechanism to optimize the load), run a single build, and remove the agent when the build is complete. Here we need to use this plugin to start the Jenkins Slave agent mirror to perform Jenkins’ Job.

4, Kubernetes Cli

Kubernetes Cli plug-in is used to provide kubectl and Kubernetes cluster interaction environment when Jenkins Job is executed. Kubectl-related commands can be allowed in Pipeline or freestyle projects. Its main role is to provide the Kubectl runtime environment, of course, can also provide the HELM runtime environment.

5. Config File Provider

The function of Config File Provider plug-in is to provide Jenkins to store properties, XML, JSON, settings. XML and other information, which can be written into the storage configuration during Pipeline execution.

For example, save a Maven global settings. XML file and import the global settings. XML during Pipeline Job execution so that Maven builds with the global settings. XML.

6, Pipeline Utility Steps

This is a plug-in that manipulates files, such as reading and writing JSON, YAMl, POM.xml, Properties, and so on. Here, this plug-in is mainly used to read the parameter Settings of pom. XML file and obtain variables to facilitate the construction of Docker image.

Third, Jenkins configuration plug-in

1. Git plug-in configuration and use

(1) Configuration credentials:

Git typically requires a credential to be configured for authentication if you are a private project, and no configuration is required if you are a public project.

Credentials -> System -> Global Credentials -> Add credentials

(2) Used in Pipeline script:

With Git plug-in pull source code, you can set pull branch, Show pull log, pull credential, pull address, and set the credentialsId to the credentialsId parameter

Reference: Jenkins. IO/doc/pipelin…

git branch: "master" ,changelog: true , credentialsId: "xxxx-xxxx-xxxx", url: "https://github.com/xxxxxx"
Copy the code

2. Docker plug-in configuration and use

(1) Function Description:

This plug-in provides the following functions:

• Records the trace of the Docker image used in FROM
• Records the trace of the Docker image running in the container
• Run the build steps in the Docker container
• Set the Docker registry endpoint for push image validation
• Set up the Docker server endpoint for remote Docker API execution

(2) Used in Pipeline script:

This plug-in will be installed by default when Jenkins is installed. Here, the plug-in is mainly used to provide a docker login environment and execute some Docker commands. Please refer to the details.

Reference: Jenkins. IO/doc/pipelin…

// This method is to set the docker warehouse address, and then select the credential ID where the user name and password are stored for verification. Note that this is valid only in this method. docker.withRegistry("https://hub.docker.com/"."xxxxx-xxxx-xxxx-xxxx") {
    echo "Build a mirror"
    def customImage = docker.build("Hub. Mydlq. Club/myproject/springboot - the helloworld: 0.0.1." ")
    echo "Push image"
    customImage.push()
    echo "Delete mirror"
    sh "Docker rmi hub. Mydlq. Club/myproject/springboot - the helloworld: 0.0.1." " 
}
Copy the code

3. Kubernetes plug-in configuration and use

(1) Configuration credentials:

Configure the kubernetes ServiceAccount Token (KUbernetes ServiceAccount Token) to connect to the Kubernetes cluster. The account permission of this credential should be set to a larger value to avoid unknown problems. After the configuration is complete, you need to set this credential in the Cloud Cloud configuration later.

(2) Cloud configuration

System Management > System Settings > Cloud

Reference: github.com/jenkinsci/k…

Here is the configuration of connecting Kubernetes cluster and starting Jenkins Slave agent.

• Name: kubernetes
• Kubernetes address: Kubernetes. Default. SVC. Cluster. The local (default within the cluster call k8s API address)
• Disable HTTPS certificate checking: Check (do not verify HTTPS)
• Credentials: Add credentials — >Secret text — >Secret Set kubernetes Token (k8S dashboard Token, etc.)
• Jenkins address: Jenkins. Mydlqcloud: 8080 / Jenkins (used for agent and Jenkins connection address, The Jenkins service address in k8S cluster is “http://jenkins service name. Jenkins namespace: Jenkins port number/Jenkins suffix”).
• Others: The default value is ok

(3) Template Configuration

Here configure the configuration of the Pod started by Jenkins Slave in kubernetes cluster. Here four mirrors will be set, respectively:

• Jenkins Slave: used to execute the Jenkins Job command.
• Helm-kubectl: used to run the Helm command.
• Docker is used to compile and push Docker images
• Maven: Used for Maven compilation and packaging.

Here, these four images are combined into a Pod to facilitate the execution of various commands for continuous deployment interactions.

Template basic configuration:

Original Yaml Settings:

In the original YAML field of Pod, fill in the following YAML file content for configuration. The following YAML configuration will be used as the basic configuration of Jenkins Slave Pod. If certain information is configured on the interface above, the values set in YAML will be automatically replaced. This is equivalent to the yamL file as a default configuration.

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: jenkins-slave
spec:
  serviceAccountName: jenkins-admin
  securityContext:                  Container security Settings
    runAsUser: 0                    Run the container as ROOT
    privileged: true                # grant privileged execution container
  containers:
  - name: jnlp                      # Jenkins Slave imageImage: registry.cn-shanghai.aliyuncs.com/mydlq/jenkins-jnlp-slave:3.27-1Set the working directory
    workingDir: /home/jenkins/agent
    tty: true
  - name: docker                    # Docker mirrorImage: registry.cn-shanghai.aliyuncs.com/mydlq/docker:18.06.2-dindcommand: ['cat']
    tty: true
    volumeMounts:
    - name: docker
      mountPath: /usr/bin/docker
    - name: docker-sock
      mountPath: /var/run/docker.sock
    - name: docker-config
      mountPath: /etc/docker
  - name: maven                     # Maven mirrorImage: registry.cn-shanghai.aliyuncs.com/mydlq/maven:3.6.0-jdk8-alpinecommand:
    - cat
    tty: true
    volumeMounts:
    - name: maven-m2
      mountPath: /root/.m2
  - name: helm-kubectl              #Kubectl & Helm mirrorImage: registry.cn-shanghai.aliyuncs.com/mydlq/helm-kubectl:2.13.1command:
    - cat
    tty: true
  volumes:
  - name: docker                    # Hang the host Docker folder into the container for easy storage & pull local image
    hostPath: 
      path: /usr/bin/docker
  - name: docker-sock               Host docker. sock into the container
    hostPath: 
      path: /var/run/docker.sock
  - name: docker-config             Hang the host Docker configuration in the entry container
    hostPath: 
      path: /etc/docker
  - name: maven-m2                  #Maven local repository is mounted to NFS shared storage for simultaneous access and storage by different nodesNFS: server: 192.168.2.11 path:"/nfs/data/maven-m2"
# nodeSelector:
# kubernetes.io/hostname: node-2-12
Copy the code

4. Kubernetes Cli plug-in configuration and use

(1) Configuration credentials:

Configure the kubernetes cluster connection credential, this credential can be the same as the above kubernetes plug-in credential, are used to connect to the Kubernetes cluster

(2) Used in Pipeline script:

The main function of this plug-in is to provide the implementation of kubectl environment Settings, in this plug-in method equivalent to kubectl, helm and other environment Settings, and then use the relevant image can execute the relevant commands.

Reference: Jenkins. IO/doc/pipelin…

// provide the kubectl execution environment, where the credential ID stored the token and kubernetes API address withKubeConfig([credentialsId:"xxxx-xxxx-xxxx-xxxx",serverUrl: "https://kubernetes.default.svc.cluster.local"]) {
    sh "kubectl get nodes"
}
Copy the code

5. Config File Provider plug-in

1. Configure Maven settings.xml

The plug-in “Config File Provider” was installed during Jenkins installation. The function of this plug-in is to store properties, XML, JSON, settings. XML and other information in Jenkins. The following list is opened here. Configure a global Maven settings. XML file.

System Management – >Managed Files – >Add a new Config – >Global Maven settings.xml

Add a global setting.xml setting that points the repository address to the Aliyun Maven repository address to speed up the jar package download.

<mirror>
    <id>alimaven</id>
    <name>aliyun maven</name>
    <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
    <mirrorOf>central</mirrorOf>
</mirror>
Copy the code

(2) Used in Pipeline script:

Reference: Jenkins. IO/doc/pipelin…

Can be used in a Pipeline script to generate the file set above as follows:

ConfigFileProvider ([configFile(fileId:"75884c5a-4ec2-4dc0-8d87-58b6b1636f8a", targetLocation: "settings.xml")]) {// The file exists only in the methodecho "cat settings.xml"
}
Copy the code

6. Pipeline Utility Steps plug-in

(1) Function description

This plug-in provides the following functions:

• Extracting Zip files
• Creating a Zip file
• Create a normal file.
• Generate a Yaml file.
• Write maven project files.
• Find files in the workspace.
• Read the properties file parameters.
• Read JSON from a file in the workspace.
• Read the Maven project’s POM.xml file
• …

(2) Used in Pipeline script:

This plugin is mainly used to read the project parameters of pom.xml for docker image compilation. Another function is to generate files, such as YAML files, HELM certificates, and so on, while the script is running.

Reference: Jenkins. IO/doc/pipelin…

// Read the pom. XML file pom =readMavenPom file: "./pom.xml"  
echo "${pom.artifactId}:${pom.version}"
Copy the code

4. Test plug-ins

In order to ensure that the plug-in is correctly configured and executed, Jenkins agent is started in kubernetes environment to execute the task, and the test will be conducted here.

1. Create pipeline tasks

Create a task named “k8S-test” of type “Pipeline”.

2. Configure pipeline tasks

(1) General configuration

• For security, forbid concurrent builds.
• In order to improve efficiency, the pipeline efficiency is set here, and the set overwrite is persisted.

(2) Assembly line script

Here is a simple script to introduce the Kubernetes plugin provided by the Pipeline method, as follows:

// Agent name, enter the label of the Template Template in Cloud set in system Settings def label ="jnlp-agent"// Call Kubernetes podTemplate(label: label,cloud:'kubernetes'){// Run the script node (label) {echo "Test Jenkins Slave agent in Kubernetes! ~"}}Copy the code

(3) Running assembly line tasks

Return to the task screen and click Construct now to execute the task.

3. View pipeline logs

Then click in the execution history bar to view the log information output by the console.

5. Preparation before deployment

1. Compare the storage locations of configuration files

The following is only my personal opinion, there is a better way, hope to inform.

In Jenkins, where is it convenient to store our Jenkinsfile script? Here I have three ideas:

• 1. Create a new Git project for storing different Jenkinsfile scripts. When Jenkins creates the task, specify the Git address for storing the scripts.
• 2. Put it into each project, read Git project when executing Jenkins task, detect Jenkinsfile script from it and execute it;
• 3. Each script was placed in the configuration of each Jenkins task, and the script set in the configuration was executed each time;

Compare the three:

• The first way is convenient for unified management. Once the configuration on Git is changed, the pipeline script of Jenkins task will change accordingly.
• The second way can be set separately for each project, more flexible, is not convenient for unified management, maintenance needs each project team;
• The third method needs to set the script in the configuration every time the new project is created, which is laborious and inconvenient to maintain, so it is not recommended.

2. Set the configuration file to the project

Here you need to store some configuration files in the project source code to read the corresponding configuration parameters in the pipeline, such as:

• SpringBoot project for testing HelloWorld.
• Dockerfile: the file used by Docker to compile the image, such as the packaged base image, etc.
• Values. Yaml: Chart configuration file for Helm startup, which sets some chart configuration information telling you how to start the application.

Github address: github.com/my-dlq/spri…

(1), Dockerfile

The FROM registry.cn-shanghai.aliyuncs.com/mydlq/openjdk:8u201-jdk-alpine3.9 VOLUME/TMP ADD target / *. Jar app. The jar RUN sh -c'touch /app.jar'
ENV JAVA_OPTS="-Xmx512M -Xms256M -Xss256k -Duser.timezone=Asia/Shanghai"
ENV APP_OPTS=""
ENTRYPOINT [ "sh"."-c"."java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar $APP_OPTS" ]
Copy the code

(2), values. Yaml

kind: Deployment
image:
  pullPolicy: "Always"
replicas: 1
resources:
  limits:
    memory: 512Mi
    cpu: 1000m
  requests:
    memory: 256Mi
    cpu: 500m

Java && app environment variable Settings
env: 
  - name: "JAVA_OPTS"
    value: "-Xmx512M -Xms355M -Xss256k -Duser.timezone=Asia/Shanghai"
  - name: "APP_OPTS"
    value: "" 

envFrom:
  #- configMapRef:
  # name: env-config

service:
  type: NodePort                     #Service type (ClusterIP, NodePort, None)
  labels: 
    svcEndpoints: actuator
  annotations: {}
  ports: 
    - name: server
      port: 8080
      targetPort: 8080
      protocol: TCP
      nodePort: 30080 
    - name: management
      port: 8081
      targetPort: 8081
      protocol: TCP
      nodePort: 30081
Copy the code

3. Test whether the operating environment is available

Here is a simple script to test whether the environment is working, such as slave image git command can be executed, maven image MVN command can be used, etc.

Container (‘ docker ‘) can be used to reference the kubernetes plugin container, using the different client functions of each container to execute the corresponding command.

Change the pipeline script from the task configuration you created earlier to the following:

def label = "jnlp-agent"

podTemplate(label: label,cloud: 'kubernetes' ){
    node (label) {
        stage('Git stage') {echo 1. Start pulling code
            sh "git version"
        }
        stage('the Maven phase'){
            container('maven') {
                echo "2, Start Maven compile, push to local library"
                sh "mvn -version"
            }
        }
        stage('Docker stage'){
            container('docker') {
                echo 3. Start reading Maven POM variables and execute Docker compilation, push, and delete.
                sh "docker version"
            }
        }
         stage('Helm stage'){
            container('helm-kubectl') {
                echo "4. Start checking Kubectl environment, test execution Helm deployment, and execute deployment"
                sh "helm version"}}}}Copy the code

The Jenkins Slave container integrates git client by default. The overall pipelined execution is in the Jenkins Slave container, and the task is executed in Jenkins Slave by default, so there is no need to set the container name.

Then run the following command to view logs:

Running on jnlp-agent-g7qk5 in/ home/Jenkins/workspace/k8s - test [Pipeline] {/ Pipeline stage [Pipeline] {phase (Git) [Pipeline]echo[Pipeline] git version git version 2.11.0 [Pipeline]} // stage [Pipeline] stage [Pipeline] [Pipeline] {[Pipeline]echo2, start Maven compilation, pushed to the local library [Pipeline] sh + MVN - version Apache Maven 3.6.0 (97 c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47z) Maven Home: /usr/share/ Maven Java version: 1.8.0_201, Vendor: Oracle Corporation, Runtime: 2018-10-24T18:41:47z) Maven Home: /usr/share/ Maven Java version: 1.8.0_201, Vendor: Oracle Corporation, Runtime: 2018-10-24T18:41:47z /usr/lib/jvm/java-1.8-openJDK /jre Default locale: en_US, platform Encoding: UTF-8 OS name:"linux", version: "3.10.0-957.1.3. El7. X86_64", arch: "amd64", family: "unix"[Pipeline] [Pipeline] [Pipeline] // stage [Pipeline] stage [Pipeline] { [Pipeline] container [Pipeline] { [Pipeline]echo[Pipeline] sh + Docker version Client: version: 18.06.2 -CE API version: 1.38 Go version: go1.10.4 Git commit: 6D37F41 Built: Sun Feb 10 03:43:40 2019 OS/Arch: Linux/AMD64 ExperimentalfalseServer: Docker Engine - Community Engine: Version: 18.09.3 API Version: 1.39 (minimum Version 1.12) Go Version: Git commit: 774a1F4 Built: Thu Feb 28 06:02:24 2019 OS/Arch: Linux/AMd64 Experimental:false[Pipeline] [Pipeline] [Pipeline] [Pipeline] [Pipeline] [Pipeline] [Pipeline] container [Pipeline] { [Pipeline]echo[Pipeline] sh + Helm Version Client: &version. version {SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.1", GitCommit:"79d07943b03aea2b76c12644b4b54733bc5958d6", GitTreeState:"clean"}
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS
Copy the code

If the execution status is SUCCESS, the environment is available. Otherwise, check the problem.

Start writing the Pipeline script

Scripting is done in stages, then tested step by step, and then merged together. Create a new task called K8S-Pipeline and enter the Pipleline script in the configuration Item script box.

1. Git pull

Here I pull a simple SpringBoot Demo project on Github for practice.

Groovy scripts

def label = "jnlp-agent"

podTemplate(label: label,cloud: 'kubernetes' ){
    node (label) {
        stage('Git stage') {echo "Git stage"
            git branch: "master" ,changelog: true , url: "https://github.com/my-dlq/springboot-helloworld.git"}}}Copy the code

Viewing Execution Logs

. Running on jnlp-agent-dhr1hin/ home/Jenkins/workspace/k8s - pipeline [pipeline] {/ pipeline stage [pipeline] {phase (Git) [pipeline]echo1, start pull code [Pipeline] sh + gitclone https://github.com/my-dlq/springboot-helloworld.git
Cloning into 'springboot-helloworld'. [Pipeline] sh + ls-l
total 0
drwxr-xr-x 4 root root 79 Apr 28 07:00 springboot-helloworld
[Pipeline] }
......
Finished: SUCCESS
Copy the code

You can see from the logs output by the console that the pull is successful. Proceed to the next step, the Maven phase.

2. Maven compilation

Here you will do Maven compilation, which compiles the Java source into a Jar project for later packaging into the Docker image. (Unit testing can also be done in Maven, but for some reasons not covered here, you can do it yourself by executing test commands.)

Groovy scripts

def label = "jnlp-agent"

podTemplate(label: label,cloud: 'kubernetes' ){
    node (label) {
        stage('Git stage') {echo "Git stage"
            git branch: "master" ,changelog: true , url: "https://github.com/my-dlq/springboot-helloworld.git"
        }
        stage('the Maven phase'){
            container('maven'ConfigFileProvider ([configFile(fileId: configFile)) {// Here we refer to the global settings. XML file set above, import it according to its ID and create this file."75884c5a-4ec2-4dc0-8d87-58b6b1636f8a", targetLocation: "settings.xml")]){
                    sh "mvn clean install -Dmaven.test.skip=true --settings settings.xml"
                }
            }
        }
    }
}
Copy the code

Viewing Execution Logs

. [INFO] -- maven-clean-plugin:3.1.0:clean (default-clean) @springboot-helloword -- [INFO] [INFO] -- Maven-resources-plugin :3.1.0:resources (default-resources) @springboot-helloword -- [INFO] Using maven-resources-plugin:3.1.0:resources (default-resources) @springboot-helloword -- [INFO] Using'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ springboot-helloword ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 sourceFiles to/home/Jenkins/workspace/k8s - pipeline/target/classes/INFO/INFO maven - resources - the plugin: 3.1.0:testResources (default-testResources) @ springboot-helloword ---
[INFO] Not copying testResources [INFO] [INFO] -- Maven-compiler-plugin :3.8.0: resources [INFO] [INFO] -- Maven-compiler-plugin :3.8.0:testCompile (default-testCompile) @ springboot-helloword ---
[INFO] Not compiling testSources [INFO] [INFO] -- Maven-Surefire-plugin :2.22.1: Sources [INFO] [INFO] -- Sources [INFO] -- Maven-Surefire-plugin :2.22.1:testTo be skipped over. (default-test) @springboot-helloword -- [INFO] Tests are skipped. [INFO] [INFO] -- Maven-jar-plugin :3.1.1:jar (default-jar) @ springboot-helloword --- [INFO] Building jar: / home/Jenkins/workspace/k8s - pipeline/target/springboot - helloword - 0.0.1. Jar [INFO] - [INFO] Spring - the boot - maven - plugin: 2.1.4. RELEASE: repackage (repackage) @ springboot - helloword - [INFO] Replacing main an artifact With repackaged archive [INFO] [INFO] -- maven-install-plugin:2.5.2:install (default-install) @springboot-helloWord - [INFO] Installing/home/Jenkins/workspace/k8s - pipeline/target/springboot - helloword - 0.0.1. Jar to / root/m2 / repository/club/mydlq/springboot - helloword / 0.0.1 / springboot - helloword - 0.0.1. Jar [INFO] Installing /home/jenkins/workspace/k8s-pipeline/pom.xml to / root/m2 / repository/club/mydlq/springboot - helloword / 0.0.1 / springboot - helloword - 0.0.1. Pom/INFO ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- [INFO] Total time: 7.989 s [INFO] Finished at: 2019-04-28T09:38:03Z [INFO] ------------------------------------------------------------------------ ...... [Pipeline] // podTemplate [Pipeline] End of Pipeline Finished: SUCCESSCopy the code

3, Docker compilation

Groovy scripts

def label = "jnlp-agent"

podTemplate(label: label,cloud: 'kubernetes' ){
    node (label) {
        stage('Git stage') {echo "Git stage"
            git branch: "master" ,changelog: true , url: "https://github.com/my-dlq/springboot-helloworld.git"
        }
        stage('the Maven phase') {echo "Maven phase"
            container('maven'ConfigFileProvider ([configFile(fileId: configFile)) {// Here we refer to the global settings. XML file set above, import it according to its ID and create this file."75884c5a-4ec2-4dc0-8d87-58b6b1636f8a", targetLocation: "settings.xml")]){
                    sh "mvn clean install -Dmaven.test.skip=true --settings settings.xml"
                }
            }
        }
        stage('Docker stage') {echo "Docker stage"
            container('docker') {// Read poM parametersecho "Read pom.xml parameters"
                pom = readMavenPom file: './pom.xml'// Set the address hub ="registry.cn-shanghai.aliyuncs.com"// Set the repository project name project_name ="mydlq"
                echo "Compile the Docker image"
                docker.withRegistry("http://${hub}"."ffb3b544-108e-4851-b747-b8a00bfe7ee0") {
                    echo "Build a mirror"Def customImage = docker.build() def customImage = docker.build() def customImage = docker.build() def customImage = docker.build()"${hub}/${project_name}/${pom.artifactId}:${pom.version}")
                    echo "Push image"
                    customImage.push()
                    echo "Delete mirror"
                    sh "docker rmi ${hub}/${project_name}/${pom.artifactId}:${pom.version}" 
                }
            }
        }
    }
}
Copy the code

Viewing Execution Logs

Compile Docker image [Pipeline] withEnv [Pipeline] {[Pipeline] withDockerRegistry Executing shell script inside container [docker] of pod [jnlp-agent-v6f1f] Executingcommand: "docker" "login" "-u" "3******[email protected]" "-p"* * * * * * * *"http://registry.cn-shanghai.aliyuncs.com"  /home/jenkins/workspace/k8s-pipeline2@tmp/b52e213b-a730-4120-b004-decd8e16b246/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[Pipeline] {
[Pipeline] echoBuild image/Pipeline sh + docker build - t registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloword:0.0.1. Sending the build Context to Docker Daemon 18.37MB Step 1/7: The FROM registry.cn-shanghai.aliyuncs.com/mydlq/openjdk:8u201-jdk-alpine3.9 -- - > 3675 b9f543c5 Step 2/7: VOLUME /tmp ---> Runningin 7fc4af80e6ce
Removing intermediate container 7fc4af80e6ce
 ---> 4e4224d3b50b
Step 3/7 : ADD target/*.jar app.jar
 ---> 0c24118d522f
Step 4/7 : RUN sh -c 'touch /app.jar'
 ---> Running in 8836cb91e1ca
Removing intermediate container 8836cb91e1ca
 ---> 389e604851b6
Step 5/7 : ENV JAVA_OPTS="-Xmx512M -Xms256M -Xss256k -Duser.timezone=Asia/Shanghai"
 ---> Running in 5126902b1e6b
Removing intermediate container 5126902b1e6b
 ---> 055ad2b9c49d
Step 6/7 : ENV APP_OPTS=""
 ---> Running in cf8ea4b61eea
Removing intermediate container cf8ea4b61eea
 ---> 07dd4fdda44a
Step 7/7 : ENTRYPOINT [ "sh"."-c"."java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar $APP_OPTS" ]
 ---> Running in 93c4d0d859e1
Removing intermediate container 93c4d0d859e1
 ---> d29e092f2c17
Successfully built d29e092f2c17
Successfully tagged registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloword:0.0.1
[Pipeline] dockerFingerprintFrom
[Pipeline] echoPush the mirror/Pipeline sh + docker tag registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloword:0.0.1 Registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloword:0.0.1 [Pipeline] sh + docker push Registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloword:0.0.1 The push refers to The repository [registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloword] 8d45ad1172aa: Preparing aba126d6a94c: Preparing a464c54f93a9: Mounted from mydlq/openjdk dee6aef5c2b6: Mounted from mydlq/openjdk aba126d6a94c: Pushed 8d45AD1172AA: Pushed 0.0.1: Digest: sha256:2c661931a3c08a1cad1562ec4936c68f06b4b3ffcec5de14c390ae793cf5b53b size: 1371 [Pipeline]echoRemove mirror/Pipeline sh + docker rmi registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloword:0.0.1 Untagged: Registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloword:0.0.1 Untagged: registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloword@sha256:2c661931a3c08a1cad1562ec4936c68f06b4b3ffcec5de14c390 ae793cf5b53b Deleted: sha256:d29e092f2c175a3662af0175e62462238583330ba7d46b84d89134056ba14027 Deleted: sha256:07dd4fdda44a12d7749e5e7f60b1943c83e6d0a3da2e4f279ce4d53f3b04f27e ...... Finished: SUCCESSCopy the code

4. Helm starts application

Create the Helm execution method

Here, the method to execute helm is created in advance and simply encapsulated for pipeline execution. This method is called to perform the corresponding HELM operation.

• Method name: helmDeploy()
• Available parameters:

parameter	describe
– init:	Whether to perform helm initialization
– url:	Initialize the chart warehouse address
– dry:	Whether the deployment is attempted
– name:	The Release name of the deployed application
– namespace:	Which Namespace does the application start to
– image:	The mirror of
– tag:	Image label
– template:	Chart template selected

Def helmDeploy(Map args) {// Helm is initializedif(args.init){
        sh "helm init --client-only --stable-repo-url ${args.url}"} // Helm attempts to deployelse if (args.dry_run) {
        println "Try Helm deployment to verify that it works."
        sh "helm upgrade --install ${args.name} --namespace ${args.namespace} -f values.yaml --set ${args.repository}.${args.tag} stable/${args.template} --dry-run --debug"} // The Helm is officially deployedelse {
        println "Official Helm deployment"
        sh "helm upgrade --install ${args.name} --namespace ${args.namespace} -f values.yaml --set ${args.repository}.${args.tag} stable/${args.template}"}} // method callstage() {
    echo "Helm initialization http://chart.mydlq.club"
    helmDeploy(init: true ,url: "Helm Warehouse Address");
    echo "Helm attempts to deploy"
    helmDeploy(init: false ,dry: true ,name: "Application name" ,namespace: "Namespace for application startup" ,image: "Mirror name",tag: "Mirror label" ,template: "Selected Chart template")
    echo "Helm officially deployed"
    helmDeploy(init: false ,dry: false ,name: "Application name" ,namespace: "Namespace for application startup" ,image: "Mirror name",tag: "Mirror label" ,template: "Selected Chart template")}Copy the code

Complete Groovy scripts

def label = "jnlp-agent"// execute the Helm method def helmDeploy(Map args) {if(args.init){
        println "Helm initialization"
        sh "helm init --client-only --stable-repo-url ${args.url}"
    } else if (args.dry_run) {
        println "Try Helm deployment to verify that it works."
        sh "helm upgrade --install ${args.name} --namespace ${args.namespace} ${args.values} --set ${args.image}.${args.tag} stable/${args.template} --dry-run --debug"
    } else {
        println "Official Helm deployment"
        sh "helm upgrade --install ${args.name} --namespace ${args.namespace} ${args.values} --set ${args.image}.${args.tag} stable/${args.template}"PodTemplate (label: label,cloud:'kubernetes' ){
    node (label) {
        stage('Git stage') {echo "Git stage"
            git branch: "master" ,changelog: true , url: "https://github.com/my-dlq/springboot-helloworld.git"
        }
        stage('the Maven phase') {echo "Maven phase"
            container('maven'ConfigFileProvider ([configFile(fileId: configFile)) {// Here we refer to the global settings. XML file set above, import it according to its ID and create this file."75884c5a-4ec2-4dc0-8d87-58b6b1636f8a", targetLocation: "settings.xml")]){
                    sh "mvn clean install -Dmaven.test.skip=true --settings settings.xml"
                }
            }
        }
        stage('Docker stage') {echo "Docker stage"
            container('docker') {// Read poM parametersecho "Read pom.xml parameters"
                pom = readMavenPom file: './pom.xml'// Set the address hub ="registry.cn-shanghai.aliyuncs.com"// Set the repository project name project_name ="mydlq"
                echo "Compile the Docker image"
                docker.withRegistry("http://${hub}"."ffb3b544-108e-4851-b747-b8a00bfe7ee0") {
                    echo "Build a mirror"Def customImage = docker.build() def customImage = docker.build() def customImage = docker.build() def customImage = docker.build()"${hub}/${project_name}/${pom.artifactId}:${pom.version}")
                    echo "Push image"
                    customImage.push()
                    echo "Delete mirror"
                    sh "docker rmi ${hub}/${project_name}/${pom.artifactId}:${pom.version}" 
                }
            }
        }
        stage('Helm stage'){
            container('helm-kubectl') {
                withKubeConfig([credentialsId: "8510eda6-e1c7-4535-81af-17626b9575f7",serverUrl: "https://kubernetes.default.svc.cluster.local"] {// set the parameter image ="image.repository=${hub}/${project_name}/${pom.artifactId}"
                    tag = "image.tag=${pom.version}"
                    template = "spring-boot"
                    repo_url = "http://chart.mydlq.club"
                    app_name = "${pom.artifactId}"// Check whether yamL file def values = exists""
                    if (fileExists('values.yaml')) {
                        values = "-f values.yaml"} // Execute the Helm methodecho "Helm initialization"
                    helmDeploy(init: true ,url: "${repo_url}");
                    echo "Helm performs deployment tests"
                    helmDeploy(init: false ,dry_run: true ,name: "${app_name}" ,namespace: "mydlqcloud" ,image: "${image}" ,tag: "${tag}" , values: "${values}" ,template: "${template}")
                    echo "Helm performs formal deployment"
                    helmDeploy(init: false ,dry_run: false ,name: "${app_name}" ,namespace: "mydlqcloud",image: "${image}" ,tag: "${tag}" , values: "${values}" ,template: "${template}")}}}}}Copy the code

Viewing Execution Logs

. Executing shell script inside container [helm-kubectl] of pod [jnlp-agent-r3c8h] Executingcommand: "kubectl" "config" "set-cluster" "k8s" "--server=https://kubernetes.default.svc.cluster.local" "--insecure-skip-tls-verify=true" 
exit
Cluster "k8s" set.
Executing shell script inside container [helm-kubectl] of pod [jnlp-agent-r3c8h]
Executing command: "kubectl" "config" "set-credentials" "cluster-admin" ******** 
Switched to context "k8s".
[Pipeline] {
[Pipeline] fileExists
[Pipeline] echoHelm initializationechoHelm initialization [Pipeline] sh + Helm init --client-only -- stables - rebo-url http://chart.mydlq.club Creating /root/.helm/repository/repositories.yaml Adding stable repo with URL: http://chart.mydlq.club AddinglocalRepo with URL: http://127.0.0.1:8879/charts$HELM_HOME has been configured at /root/.helm.
Not installing Tiller due to 'client-only' flag having been set
Happy Helming!
[Pipeline] echoHelm performs deployment testsecho[Pipeline] sh + Helm upgrade --install springboot-helloworld --namespace mydlqCloud-f values.yaml --set 'image.repository=registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloworld, image. The tag = 0.0.1' stable/spring-boot --dry-run --debug
[debug] Created tunnel using local port: '37001'

[debug] SERVER: "127.0.0.1:37001"[the debug] Fetched stable/spring - the boot to/root /. Helm/cache/archive/spring - the boot - 1.0.4. TGZ Release"springboot-helloworld"Does not exist. Installing it now. [the debug] CHART PATH: / root /. Helm/cache/archive/spring - the boot - 1.0.4. TGZ [Pipeline]echoHelm performs formal deploymentechoSh + Helm Upgrade --install springboot-helloworld --namespace mydlqCloud-f values.yaml --set 'image.repository=registry.cn-shanghai.aliyuncs.com/mydlq/springboot-helloworld, image. The tag = 0.0.1' stable/spring-boot
Release "springboot-helloworld" does not exist. Installing it now.
NAME:   springboot-helloworld
LAST DEPLOYED: Mon Apr 29 07:31:39 2019
NAMESPACE: mydlqcloud
STATUS: DEPLOYED

RESOURCES:
==> v1/Pod(related)
NAME                                    READY  STATUS             RESTARTS  AGE
springboot-helloworld-7cd66cf74d-vfjr6 0/1 ContainerCreating 0 0s ==> v1/Service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE Springboot - the helloworld NodePort 10.10.87.61 < none > 8080:30080 / TCP, 8081:30081 / TCP 0 s = = > v1beta1 / Deployment NAME READY UP-TO-DATE AVAILABLE AGE springboot-helloworld 0/1 1 0 0s ...... Finished: SUCCESSCopy the code

5. Test interface

After the execution of Helm step above, a simple test can be carried out. Chart referenced by this project is a simple SpringBoot project, in which two ports, 30080 & 30081, are exposed in NodePort mode, corresponding to 8080 and 8081 respectively. A Hello World interface is provided as “/ Hello”, so let’s access this interface address here:

http://192.168.2.11:30080/hello

Seven, improve the Pipeline script

1. Set the timeout period

Set a task timeout period. If the task is not completed within the timeout period, the operation fails.

Format:

timeout(time: 20, unit: 'SECONDS') {// pipeline code}Copy the code

Example:

Set the timeout period to 60 seconds for the Jenkins Slave node to execute the task.

def label = "jnlp-agent"
timeout(time: 60, unit: 'SECONDS') {
    podTemplate(label: label,cloud: 'kubernetes' ){
        node (label) {
            stage('Git stage') {echo "Git stage"
            }
            stage('the Maven phase') {echo "Maven phase"
            }
            stage('Docker stage') {echo "Docker stage"
            }
            stage('Helm stage') {echo "Helm stage"}}}}Copy the code

2. Set email notification

(1) Set the mailbox enable POP3/SMTP/IMAP Settings

This is p.O. Box 163

2. Install the Email Extension Template plugin

The plug-in “Email Extension Template” is installed here to set up the mail Template.

(3) Set the default mail parameters

System Management -> System Settings:

Configure Jenkins Location and Extended E-mail Notification, where the email address of the system administrator must be consistent with the value of User Name.

Jenkins Location setting:

The parameter name	describe
– Jenkins URL:	Jenkins address, used to write into the content when sending emails
– Email address of system administrator:	Mail Server Account

Extended Email Notification Settings:

The parameter name	describe
– the SMTP server:	SMTP email address
-default user E-mail suffix:	Mail server suffix
– User Name:	Mail Server Account
– the Password:	SMTP authorization code of the mail server
– Default Content Type:	Set the email text format
– Enable Debug Mode:	Enabling the Debug mode

(4) Create assembly line projects

Create a Pipeline project, used to write a Pipeline script test mail send, and configure the Pipeline script, here write a simple Pipeline script, call the Emailext method to execute the send mail.

Script content:

def label = "jnlp-agent"
podTemplate(label: label,cloud: 'kubernetes' ){
    node (label) {
        stage('Git stage') {echo "Git stage"
        }
        stage('the Maven phase') {echo "Maven phase"
        }
        stage('Docker stage') {echo "Docker stage"
        }
        stage('Helm stage') {echo "Helm stage"
        }
        stage('email') {echo "Test send mail"
            emailext(subject: 'Task execution failed',to: '324******[email protected]',body: ' ''Test email content... '' ')}}}Copy the code

(5) Run the project to view the log

View the logs to see if the send operation was performed and the health.

Started by user admin
Running in Durability level: PERFORMANCE_OPTIMIZED
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/email-test
[Pipeline] {
[Pipeline] stage
[Pipeline] { (email)
[Pipeline] echoEmailext Sending email to: 32*****[email protected] [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESSCopy the code

(6) Check email

You can view the email information from the specified mailbox.

3. Judge success and failure to send the email

(1) Judge the success or failure of assembly line process

Here, try, catch and finally are added for assembly line. When finally is executed, whether the task is successfully constructed is judged. If it is successful, a successful email notification will be sent. If the failure occurs, an email notification is sent.

try{
    def label = "jnlp-agent"
    podTemplate(label: label,cloud: 'kubernetes' ){
        node (label) {
            stage('Git stage') {echo "Git stage"
            }
            stage('the Maven phase') {echo "Maven phase"
            }
            stage('Docker stage') {echo "Docker stage"
            }
            stage('Helm stage') {echo "Helm stage"
            }
        }
    }
}catch(Exception e) {
    currentBuild.result = "FAILURE"Def currResult = currentbuild.result? :'SUCCESS'// Determine the task status and send the message stage('email') {if (currResult == 'SUCCESS') {
            echo "Email sent successfully"
            emailext(subject: 'Task performed successfully',to: '32*****[email protected]',body: ' ''The task has successfully built... '' ')}else {
            echo "Failed to send message"
            emailext(subject: 'Task execution failed',to: '32*****[email protected]',body: ' ''Task execution failed Build failed... '' ')}}}Copy the code

(2) Send an email when the test succeeds or fails

Success:

Send the email after running the process successfully.

Failure:

Simulate intentionally performing error sending messages.

4. Put the script into the project

Put the script in the project so that you can set the Git address of the project directly when you call it.

Viii. Complete code

The complete code is as follows:

// execute the Helm method def helmDeploy(Map args) {if(args.init){
        println "Helm initialization"
        sh "helm init --client-only --stable-repo-url ${args.url}"
    } else if (args.dry_run) {
        println "Try Helm deployment to verify that it works."
        sh "helm upgrade --install ${args.name} --namespace ${args.namespace} ${args.values} --set ${args.image}.${args.tag} stable/${args.template} --dry-run --debug"
    } else {
        println "Official Helm deployment"
        sh "helm upgrade --install ${args.name} --namespace ${args.namespace} ${args.values} --set ${args.image}.${args.tag} stable/${args.template}"}} // Jenkins slave Execute the slave task timeout(time: 600, unit:'SECONDS') {
    try{
        def label = "jnlp-agent"
        podTemplate(label: label,cloud: 'kubernetes' ){
            node (label) {
                stage('Git stage') {echo "Git stage"
                    git branch: "master" ,changelog: true , url: "https://github.com/my-dlq/springboot-helloworld.git"
                }
                stage('the Maven phase') {echo "Maven phase"
                    container('maven'ConfigFileProvider ([configFile(fileId: configFile)) {// Here we refer to the global settings. XML file set above, import it according to its ID and create this file."75884c5a-4ec2-4dc0-8d87-58b6b1636f8a", targetLocation: "settings.xml")]){
                            sh "mvn clean install -Dmaven.test.skip=true --settings settings.xml"
                        }
                    }
                }
                stage('Docker stage') {echo "Docker stage"
                    container('docker') {// Read poM parametersecho "Read pom.xml parameters"
                        pom = readMavenPom file: './pom.xml'// Set the address hub ="registry.cn-shanghai.aliyuncs.com"// Set the repository project name project_name ="mydlq"
                        echo "Compile the Docker image"
                        docker.withRegistry("http://${hub}"."ffb3b544-108e-4851-b747-b8a00bfe7ee0") {
                            echo "Build a mirror"Def customImage = docker.build() def customImage = docker.build() def customImage = docker.build() def customImage = docker.build()"${hub}/${project_name}/${pom.artifactId}:${pom.version}")
                            echo "Push image"
                            customImage.push()
                            echo "Delete mirror"
                            sh "docker rmi ${hub}/${project_name}/${pom.artifactId}:${pom.version}" 
                        }
                    }
                }
                stage('Helm stage'){
                    container('helm-kubectl') {
                        withKubeConfig([credentialsId: "8510eda6-e1c7-4535-81af-17626b9575f7",serverUrl: "https://kubernetes.default.svc.cluster.local"] {// set the parameter image ="image.repository=${hub}/${project_name}/${pom.artifactId}"
                            tag = "image.tag=${pom.version}"
                            template = "spring-boot"
                            repo_url = "http://chart.mydlq.club"
                            app_name = "${pom.artifactId}"// Check whether yamL file def values = exists""
                            if (fileExists('values.yaml')) {
                                values = "-f values.yaml"} // Execute the Helm methodecho "Helm initialization"
                            helmDeploy(init: true ,url: "${repo_url}");
                            echo "Helm performs deployment tests"
                            helmDeploy(init: false ,dry_run: true ,name: "${app_name}" ,namespace: "mydlqcloud" ,image: "${image}" ,tag: "${tag}" , values: "${values}" ,template: "${template}")
                            echo "Helm performs formal deployment"
                            helmDeploy(init: false ,dry_run: false ,name: "${app_name}" ,namespace: "mydlqcloud",image: "${image}" ,tag: "${tag}" , values: "${values}" ,template: "${template}")
                        }
                    }
                }
            }
        }
    }catch(Exception e) {
        currentBuild.result = "FAILURE"Def currResult = currentbuild.result? :'SUCCESS'// Determine the task status and send the message stage('email') {if (currResult == 'SUCCESS') {
                echo "Email sent successfully"
                emailext(subject: 'Task performed successfully',to: '32******[email protected]',body: ' ''The task has successfully built... '' ')}else {
                echo "Failed to send message"
                emailext(subject: 'Task execution failed',to: '32******[email protected]',body: ' ''Task execution failed Build failed... '' ')}}}}Copy the code