preface

In the middle of this month, Jenkins Operator officially became a sub-project of Jenkins [1], which will go a long way toward bridging the gap between Jenkins and Kubernetes.

Operators are an extension mechanism of Kubernetes that users can use to make their applications run on Kubernetes native (K8S native). More details about operators can be found in the official Kubernetes documentation [2].

Jenkins is an open source continuous delivery tool with a strong community, rich APIs & plug-ins, and a large audience. To enable Jenkins to run better on Kubernetes, the Jenkins community provides the Jenkins Operator. Jenkins Operators are built on the basis of immutability and declarative configuration, or code. Additional information about operators can be found on the Jenkins Operator GitHub and on the Jenkins Operator website documentation [3].

The code involved in the following demo is all in
https://github.com/majinghe/jenkins-operator.gitLibrary, you must download the library code to the local, and then perform the following operations.

Introduction: Introduction to and installation of jenkins-operator

The installation

Prerequisites for installation:

  • A Kubernetes cluster version 1.11+
  • The version of Kubectl is 1.11+

Step 1: Jenkins CRD creation

Execute the following command to create the Jenkins CRD

$ kubectl apply -f jenkins_crd.yaml
customresourcedefinition.apiextensions.k8s.io/jenkins.jenkins.io created
customresourcedefinition.apiextensions.k8s.io/jenkinsimages.jenkins.io created

Step 2: Install the Jenkins Operator

This Operator is the core of the Jenkins-operator. The main purpose of this Operator is to manage resources such as Jenkins, including creation, deletion, etc., as we’ll see later. There are two ways to install this Operator:

  • withkubectlTo complete the installation
  • withhelmTo complete the installation

About the different use commands of the two ways, you can check the official website, and this paper chooses to complete with kubectl. Start by creating a namespace:

$ kubectl create ns jenkins$ kubectl create ns jenkins

Execute the operator installation command:

$  kubectl  -n jenkins apply -f jenkins_operator.yaml
serviceaccount/jenkins-operator created
role.rbac.authorization.k8s.io/jenkins-operator created
rolebinding.rbac.authorization.k8s.io/jenkins-operator created
deployment.apps/jenkins-operator created

Check out the pod under Jenkins Namespace:

$ kubectl -n jenkins get pods -w
NAME                                READY   STATUS    RESTARTS   AGE
jenkins-operator-548d76f664-hp6pm   0/1     Pending   0          0s
jenkins-operator-548d76f664-hp6pm   0/1     ContainerCreating   0          1s
jenkins-operator-548d76f664-hp6pm   1/1     Running             0          34s

Step 3: Create the Jenkins instance

If a Jenkins instance is created at this point, the Operator will listen for this event and create the instance based on our description of the Jenkins instance (the description file for the Jenkins instance, in YAML format). The format content of the description file for the instance can be viewed here. The documents and contents used in this paper are explained as follows:

apiVersion: jenkins.io/v1alpha2 kind: Jenkins metadata: name: jenkins spec: /* ConfigurationAscode is used to describe the configuration of the Jenkins Configrate part, such as GitHub Server, Slack, LDAP, etc. The above is organized in the form of Jenkins Configuration As Code. */ ConfigurationAsCode: Configurations: -name: Jenkins-config serviceAccount: annotations: Kubernetes. IO /service-account: Jenkins master: basePlugins: /* Configure the required plugins to install */ -name: Kubernetes version: "1.29.2" plugins: /* Configure any other plugins you want to install */ -name: LDAP version: "2.4" -name: GitHub version: "1.33.1" containers: /* Configurations for master containers, including mirrors, resource constraints, environment variables, etc. */ -Name: Jenkins-master image: jenkins/jenkins:lts imagePullPolicy: Always env: - name: JENKINS_HOME value: /var/lib/jenkins resources: limits: cpu: 1500m memory: 2Gi requests: cpu: "1" memory: 500Mi

Jenkins-config config config config config config config config config config config config config config config config config config config config config config config config config config config This content describes the configuration of Jenkins Configuration in the form of ConfigMap, such as LDAP, GitHub Server, etc. Specific content can be referred to as follows:

apiVersion: v1 kind: ConfigMap metadata: name: jenkins-config data: jenkins.yaml: | security: scriptApproval: /*Script Approval section */ ApprovedSignatures: - "method groovy.jsonslurper parse java.io.Reader" unclassified: SlackNotifier: /* Slack configuration section */ TeamDomain: slack-test TokenCredentialID: "slack-token" githubpluginConfig: / * making Server configuration section * / configs: - name: "making" apiUrl: "https://api.github.com/v3/" credentialsId: "Github -token" manageHooks: true Jenkins: clouds: /*kubernetes configuration part */ -kubernetes: jenkinsTunnel: "jenkins-operator-slave-jenkins.jenkins.svc.cluster.local:50000" jenkinsUrl: "http://jenkins-operator-http-jenkins.jenkins.svc.cluster.local:8080" name: "kubernetes" namespace: "jenkins" retentionTimeout: 15 serverUrl: "https://kubernetes.default.svc.cluster.local:443" systemMessage: */ Configurations: -server: "Configurations:" */ Configurations: -server: "" Cloud Native DevSecOps>" "xxxx" userSearchBase: "xxxx" userSearch: "xxxx" groupSearchBase: "xxxxxx" groupMembershipStrategy: fromGroupSearch: filter: ""

Then simply write the above into the YAML file to create configMap:

$ kubectl -n  jenkins apply -f config.yaml
configmap/jenkins-config created

Next, create a Jenkins instance:

$ kubectl -n jenkins apply -f jenkins.yaml
jenkins.jenkins.io/jenkins created

At this point, you can see the following information in the log of operator pod:

021-04-04T08: 21:44.049z Info Controller - Jenkins Base/POD. Go :159 Creating a new Jenkins Master Pod jenkins/jenkins-jenkins {"cr": "jenkins"}

If the operator has caught the event to create a Jenkins instance and then creates it according to the description of the instance, see the pod below the Jenkins Namespace:

$ kubectl -n jenkins get pods -w
NAME                                     READY   STATUS    RESTARTS   AGE
jenkins-jenkins                          2/2     Running   0          13d
jenkins-operator-548d76f664-hp6pm        1/1     Running   0          13d

This is a new Jenkins-Jenkins POD. This is a new Jenkins Master POD.

If you want to expose Jenkins’ service via Ingress, you can create an Ingress resource using the following YAML file:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: jenkins
 annotations:
   kubernetes.io/ingress.class: "nginx"
   nginx.ingress.kubernetes.io/redirect-to-https: "True"
spec:
 tls:
 - hosts:
   - xiaomage.devops.com
   secretName: jenkins-tls
 rules:
 - host: xiaomage.devops.com
   http:
     paths:
     - path: /
       pathType: Prefix
       backend:
         service:
           name: jenkins-operator-http-jenkins
           port:
             number: 8080

Then by visiting https://xiaomage.devops.com to access the Jenkins interface. The login username and password can be obtained by using the following command:

$ kubectl -n jenkins get secrets jenkins-operator-credentials-jenkins -o jsonpath='{.data.user}' | base64 -D
$ kubectl -n jenkins get secrets jenkins-operator-credentials-jenkins -o jsonpath='{.data.password}' | base64 -D

Jenkins can also be used in port-forward:

$ kubectl -n jenkins port-forward pods/jenkins-jenkins 8080:8080

Go directly to http://localhost:8080 to access the Jenkins interface. Get the login user name and password as above.

At this point, the process of installing Jenkins via Jenkins-operator is complete, and the next step is the Use section.

Advanced Part: Use

The traditional way to use it is to click Create Jenkins Job on the screen, configure it, and then use it again. But Jenkins-operator provides another operator — Seed Jobs, which, as the name implies, is capable of automatic discovery of Jobs. The idea behind it is to use Jenkins Job DSL and Configuration As Code: The job is described via DSL (description including job name, configuration, Pipeline script, etc.), and then the code of this description is stored on GitHub. Seed Jobs completes the job by automatically capturing the actions added by the job based on the configuration.

The Job definition file and the Job pipeline file must have the following file directory structure:

CICD/Bass Exercises ─ Jobs │ ├── Job - DSL - File ├── Pipeline - File

Seed Job can be enabled by adding the following to Jenkins’ configuration file:

API version: Jenkins. IO /v1alpha2 kind: Jenkins metadata: name: Jenkins spec: master: # Demo targets: "CICD /jobs/ demo_Pipeline.groovy" # Job DSL script position CredentialType: basicsshuserprivateKey CredentialID: github-ssh-key description: "CI/CD Repo" repositoryBranch: main repositoryUrl: [email protected]: majinghe/Jenkins - operator. Git # cicd warehouse address

Groovy describes the configuration of the demo job as follows:

#! /usr/bin/env groovy pipelineJob('Demo') {# displayName('Demo') # build history logRotator {numToKeep(10) DaystoKeep (30)} # build parameter, Parameters {choiceParam('ENV', ['SVT1', 'SVT2']) stringParam('Release', 'v1.0.0', 'Please input your release number! ')} # Pipeline Groovy {cpsScm {SCM {git {remote { url('[email protected]:majinghe/jenkins-operator.git') credentials('github-ssh-key') } branches('*/main') } } # pipeline Groovy in the library position scriptPath (' cicd/pipelines/demo. Groovy ')}}}

The job of a DSL describes a job, there are two build parameters cicd/pipelines/demo. Groovy content as follows:

def label = "jnlp-${UUID.randomUUID().toString()}" podTemplate( label: label, serviceAccount: "jenkins", namespace: "Jenkins ", containers: [containerTemplate(name: 'JNLP ', image: 'gbyukg/docker-jnlp-slave:1.0', args: '${computer.jnlpmac} ${computer.name}' ) ] ) { node(label) { container('jnlp'){ stage("Hello World"){ println "This is xiaomage, focus on Cloud Native DevSecOps!!!" }}}}

This is Xiaomage, focus on Cloud Native DevSecOps!! Then create a new Jenkins resource:

$ kubectl -n jenkins apply -f jenkins.yaml

You can then see the seed job POD under Jenkins Namespace:

$ kubectl -n jenkins get pods
NAME                                      READY   STATUS    RESTARTS   AGE
jenkins-jenkins                           1/1     Running   0          5d # jenkins master pod
jenkins-operator-548d76f664-hp6pm         1/1     Running   0          5d # jenkins operator pod
seed-job-agent-jenkins-7ff6d479db-6gqjl   1/1     Running   0          5d # see job pod

Log back into Jenkins and you’ll see two jobs, one for the Seed job and one for the actual job to be used in the end.

After that, whenever the job changes, just modify the code on GitHub for the job, and then re-run the Seed Job to update the content that actually uses the job. For the above demo job, see the build log:

You can see that the above build output is consistent with what is defined in the Pipeline. And that’s actually what it’s all about: code. If Jenkins has any problems, you can also rebuild to quickly pull up the job. If you need to backup your job history, you can either persist the job history directory or use Jenkins-Operator’s backup and restore mechanisms. See here [4] for details.

High-level: Deploy the Jenkins-Operator using Kustomize + SOPS (GPG)

The above process shows you how to install and use the jenkins-operator step by step, but creating the required resources one by one through Kubectl Apply is cumbersome and does not make sense in the context of multiple sets of differentiation. So this paper uses Kustomize to manage numerous YAML files in a differentiated environment, with the directory structure as follows:

. - Base │ ├── config. YAML │ ├── Jinks-RBac. YAML │ ├─ Jinks-RBac. YAML │ ├─ Jinks-RBac YAML │ ├─ Kustomization. YAML ├─ Secret ├─ Secret Credit.Secret. YAML ├─ Credit.Secret - dev │ ├─ ingress. YAML │ ├─ Jenkins. YAML │ ├─ kops-secret. YAML │ ─ kustomization ├─ secret. TLS. YAML ├─ Anti-Flag ─ Ingress. YAML ├─ Kops - Secret. YAML ├─ Kustomization Secret, sigma ── secret, sigma ── secret, sigma ── SVT, sigma ─ ingress. Sigma ─ Jenkins. Sigma ─ kops, sigma ─ kustomization Secret └ ─ ─ secret. TLS. Yaml

For the usage of kustomize, you can see here [5]. The entire code base above is here [6]. Sops [7] is used in this paper to encrypt sensitive information in YAML files, so that everything can be truly codified and then hosted on GitHub.

You can use the following command to quickly install the Jenkins Operator after modifying the configuration file depending on the environment

$ kustomize build --enable-alpha-plugins . | kubectl -n jenkins apply -f -
serviceaccount/jenkins unchanged
serviceaccount/jenkins-operator unchanged
role.rbac.authorization.k8s.io/jenkins-operator unchanged
rolebinding.rbac.authorization.k8s.io/jenkins-operator unchanged
clusterrolebinding.rbac.authorization.k8s.io/jenkins-role-binding unchanged
configmap/jenkins-config configured
secret/github-ssh-key configured
secret/github-token configured
secret/slack-token configured
secret/your-container-registry configured
deployment.apps/jenkins-operator configured
jenkins.jenkins.io/jenkins configured
ingress.networking.k8s.io/jenkins unchanged

–enable-alpha-plugins is used to encrypt sensitive information using SOPs. Stay tuned for future articles on how to use SOPs in more detail.

reference

[1] https://www.jenkins.io/blog/2…

[2] https://kubernetes.io/docs/co…

[3] https://github.com/jenkinsci/…

[4] https://jenkinsci.github.io/k…

[5] https://kustomize.io/

[6] https://github.com/majinghe/j…

[7] https://github.com/mozilla/sops

Source: Devsecops Sig


Author: Little horse brother

Every Thursday in May at 8 p.m., quality and testing special. Public message “quality” can obtain the address

  • 0506 Zhu Shaomin “How to Maximize Software Testing Efficiency”
  • 0513 Chen Qi “Data Driven Test”
  • Chen Ji “yes, to QA is the most effective way to improve quality!”
  • Shi Huibin “Continuous Testing of DevOps Practice”