Now the CICD of this unit is quite chaotic, and then it occurred to me that I wanted to transform it, so I made a simple assembly line with Pipeline, and the following is some introduction about it
Write a simple pipeline
Probably is such a process is simply: pull code — “compile” — “push mirror -” deployment to K8S, the following pipeline is carried out on the main line, according to the situation to increase
pipeline {
agent { label 'pdc&&jdk8' }
environment {
git_addr = "Code repository address"
git_auth = "Authentication ID when pulling code"
pom_dir = "Directory location (relative path) of POM files"
server_name = "Service Name"
namespace_name = "Namespace where the service resides"
img_domain = "Mirror address"
img_addr = "${img_domain}/cloudt-safe/${server_name}"
// cluster_name = "cluster name"
}
stages {
stage('Clear dir') {
steps {
deleteDir()
}
}
stage('Pull server code and ops code') {
parallel {
stage('Pull server code') {
steps {
script {
checkout(
[
$class: 'GitSCM'.branches: [[name: '${Branch}']],
userRemoteConfigs: [[credentialsId: "${git_auth}".url: "${git_addr}"]]
]
)
}
}
}
stage('Pull ops code') {
steps {
script {
checkout(
[
$class: 'GitSCM'.branches: [[name: 'pipeline - 0.0.1']], // Pull a branch of the build script
doGenerateSubmoduleConfigurations: false.extensions: [[$class: 'RelativeTargetDirectory'.relativeTargetDir: 'DEPLOYJAVA']], //DEPLOYJAVA: places code in this directory
userRemoteConfigs: [[credentialsId: 'chenf-o'.url: 'Build script repository address']]
]
)
}
}
}
}
}
stage('Set Env') {
steps {
script {
date_time = sh(script: "date +%Y%m%d%H%M".returnStdout: true).trim()
git_cm_id = sh(script: "git rev-parse --short HEAD".returnStdout: true).trim()
whole_img_addr = "${img_addr}:${date_time}_${git_cm_id}"
}
}
}
stage('Complie Code') {
steps {
script {
withMaven(maven: 'maven_latest_linux') {
sh "mvn -U package -am -amd -P${env_name} -pl ${pom_dir}"
}
}
}
}
stage('Build image') {
steps {
script {
dir("${env.WORKSPACE}/${pom_dir}") {
sh """ echo 'FROM base image '> Dockerfile
withCredentials([usernamePassword(credentialsId: 'faabc5e8-9587-4679-8c7e-54713ab5cd51'.passwordVariable: 'img_pwd'.usernameVariable: 'img_user')]) {
sh """ docker login -u ${img_user} -p ${img_pwd} ${img_domain} docker build -t ${img_addr}:${date_time}_${git_cm_id} . docker push ${whole_img_addr} """
}
}
}
}
}
stage('Deploy img to K8S') {
steps {
script {
dir('DEPLOYJAVA/deploy') {
// Execute the build script
sh """ /usr/local/python3/bin/python3 deploy.py -n ${server_name} -s ${namespace_name} -i ${whole_img_addr} -c ${cluster_name} """}}}// If the script fails to execute, the mirror will be deleted
post {
failure {
sh "docker rmi -f ${whole_img_addr}"
}
}
}
stage('Clear somethings') {
steps {
script {
// Delete the mirror
sh "docker rmi -f ${whole_img_addr}"
}
}
post {
success {
// If the preceding phase succeeds, the current directory will be deleted
deleteDir()
}
}
}
}
}
Copy the codeOptimize the build image
There is a command in the above pipeline is generated Dockerfile, here to do a lot of optimization, although my Dockerfile wrote a FROM, but after this will perform a series of operations, below we do not optimize the Dockerfile is not optimized
FROM base image address RUN mkdir XXXXX COPY *.jar /usr/app/app.jar ENTRYPOINT Java -jar app.jarCopy the codeThe optimized
FROM Address of a base mirrorCopy the codeDockerfile is optimized for this line… ONBUILD is an image that is based on the Java language. This image has two parts: one is the base image A containing the JDK, and the other is the image B containing the JAR package. The relationship is that A comes before B, which means that B depends on A. Assuming a complete Java-based CICD scenario, we need to pull the code, compile, image, push the image, update the POD, and in the process of image, we need to COPY the compiled product JAR package into the base image, which causes, We also need to write a Dockerfile to COPY jar packages, like this:
Jar /usr/bin/app.jar ENTRYPOINT Java -jar app.jarCopy the codeSo that looks good, basically three lines, but why use three lines when you can use one line?
Jar /*.jar /usr/bin/app.jar CMD ["/start.sh"]
Copy the codeFor example, if the image name is java-service:jdk1.8, ONBUILD will not be executed during the local mirroring process, but will be executed during the next reference
The FROM Java - service: jdk1.8Copy the codeThis line is all that is needed, and it looks much cleaner, and pipeline looks much more formal, so that every Java service can use this line of Dockerfile.
Using the credentials
Sometimes we need to authenticate when using Docker for push image. If we write directly in pipeline, it is not safe, so we have to desensitize. In this case, we need to use credentials, adding credentials is also very simple, because we only save our user name and password. So use credentials of type Username with Password, as shown below
For example, to pull git repository code, add a credential that corresponds to the following:
stage('Pull server code') {
steps {
script {
checkout(
[
$class: 'GitSCM'.branches: [[name: '${Branch}']],
userRemoteConfigs: [[credentialsId: "${git_auth}".url: "${git_addr}"[]])}}}Copy the codeThe variable ${git_auth} is the ID set when the credential is added, or an ID will be randomly generated if it is not set
thendocker pushAdd credentials in the same way as above, but we can use the syntax of Pipeline to generate one, as follows: clickPipeline Syntax
Select withCredentials: Bind credentials to variables
Then bind to the previously added credentials, select type: Username and Password (Separated)
Set the variable names for the user name and password, and then select the credentials you just added
Click generate, which is the following paragraph in pipeline above:
withCredentials([usernamePassword(credentialsId: 'faabc5e8-9587-4679-8c7e-54713ab5cd51'.passwordVariable: 'img_pwd'.usernameVariable: 'img_user')]) {
sh """ docker login -u ${img_user} -p ${img_pwd} ${img_domain} docker build -t ${img_addr}:${date_time}_${git_cm_id} . docker push ${whole_img_addr} """
}
Copy the codeCredentialsId: This ID is the randomly generated ID
Run the script to update the image
Here is the use of Python to write a small script to call kubernetes interface to do a patch operation completed. Take a look at the directory structure of this script
Core code:deploy.py
Core documents:config.yamlStorage is KubeconFIG file, and Kubernetes authentication
Here is the deploy.py script for your reference:
import os
import argparse
from kubernetes import client, config
class deployServer:
def __init__(self, kubeconfig) :
self.kubeconfig = kubeconfig
config.kube_config.load_kube_config(config_file=self.kubeconfig)
self._AppsV1Api = client.AppsV1Api()
self._CoreV1Api = client.CoreV1Api()
self._ExtensionsV1beta1Api = client.ExtensionsV1beta1Api()
def deploy_deploy(self, deploy_namespace, deploy_name, deploy_img=None, deploy_which=1) :
try:
old_deploy = self._AppsV1Api.read_namespaced_deployment(
name=deploy_name,
namespace=deploy_namespace,
)
old_deploy_container = old_deploy.spec.template.spec.containers
pod_num = len(old_deploy_container)
if deploy_which == 1:
pod_name = old_deploy_container[0].name
old_img = old_deploy_container[0].image
print("Get information for previous version \n")
print("Current Deployment has {} pod: {}\n".format(pod_num, pod_name))
print("Last version of the mirror address: {}\n".format(old_img))
print("Mirror address for this build: {}\n".format(deploy_img))
print("Replacing the mirror address of the current service.... \n")
old_deploy_container[deploy_which - 1].image = deploy_img
else:
print("Only one mirror address can be replaced")
exit(-1)
new_deploy = self._AppsV1Api.patch_namespaced_deployment(
name=deploy_name,
namespace=deploy_namespace,
body=old_deploy
)
print("Mirror address has been replaced \n")
return new_deploy
except Exception as e:
print(e)
def run() :
parser = argparse.ArgumentParser()
parser.add_argument('-n'.'--name'.help="Built service name")
parser.add_argument('-s'.'--namespace'.help="The namespace in which the service to be built")
parser.add_argument('-i'.'--img'.help="Mirror address for this build")
parser.add_argument('-c'.'--cluster'.help="Name of the cluster in which the current service resides in Rancher")
args = parser.parse_args()
if not os.path.exists('.. /config/' + args.cluster):
print("Current cluster name not set or incorrect: {}".format(args.cluster), 'red')
exit(-1)
else:
kubeconfig_file = '.. /config/' + args.cluster + '/' + 'config.yaml'
if os.path.exists(kubeconfig_file):
cli = deployServer(kubeconfig_file)
cli.deploy_deploy(
deploy_namespace=args.namespace,
deploy_name=args.name,
deploy_img=args.img
)
else:
print("Kubeconfig for current cluster does not exist, please configure it, config. Yaml under {} (note: config..format(args.cluster),
'red')
exit(-1)
if __name__ == '__main__':
run()
Copy the codeWrite relatively simple, there is no difficult place, the key place is:
new_deploy = self._AppsV1Api.patch_namespaced_deployment(
name=deploy_name,
namespace=deploy_namespace,
body=old_deploy
)
Copy the codeThis is the patch operation. Replace the content of the new mirror address and patch it. Then it’s just execution.
other
One thing to note here is that the pipeline added an exception catch as follows:
post {
success {
// If the preceding phase succeeds, the current directory will be deleted
deleteDir()
}
}
Copy the codeLife type pipeline and script type pipeline exception capture writing is different, declarative writing is to use post to judge, relatively simple, you can refer to the official document
There is also a place where parallel execution is used, where both the service code and the build script code are pulled together, to speed up the pipeline, as shown below:
parallel {
stage('Pull server code') {
steps {
script {
checkout(
[
$class: 'GitSCM'.branches: [[name: '${Branch}']],
userRemoteConfigs: [[credentialsId: "${git_auth}".url: "${git_addr}"]]
]
)
}
}
}
stage('Pull ops code') {
steps {
script {
checkout(
[
$class: 'GitSCM'.branches: [[name: 'pipeline - 0.0.1']], // Pull a branch of the build script
doGenerateSubmoduleConfigurations: false.extensions: [[$class: 'RelativeTargetDirectory'.relativeTargetDir: 'DEPLOYJAVA']], //DEPLOYJAVA: places code in this directory
userRemoteConfigs: [[credentialsId: 'chenf-o'.url: 'Build script repository address'[]])}}}}Copy the codeWell, that’s the case. A simple pipeline is done. If you want to use the pipeline quickly to complete CICD, you can refer to this article.