KubeSphere + Argo CD, true GitOps!

Shared by community user Willqy

Argo CD profile,

Argo CD is a declarative GitOps continuous delivery tool for Kubernetes. Application definitions, configurations, and environments should be declarative and version-controlled. Application deployment and lifecycle management should be automated, audit able, and easy to understand.

The Argo CD follows the GitOps pattern, which uses a Git repository as a real source for defining the required application state.

The Argo CD automatically deploits the required application state in a specified target environment. Application deployment can track updates to branches, tags, or fixed to a specific version of a manifest at Git commit time.

Liverpoolfc.tv: argoproj. Making. IO /

Argo CD architecture diagram:

The Argo CD is implemented as a Kubernetes controller that continuously monitors the running application and compares the current active state to the desired target state (specified in the Git repository). The Argo CD will treat deployed applications as OutOfSync when their running state deviates from the target state.

The Argo CD reports and visualizes differences while providing the ability to automatically or manually synchronize real-time state back to the desired target state. Any changes made to the desired target state in the Git repository can be automatically applied and synchronized to the specified target environment.

The Kubernetes configuration list supported by Argo CD includes Helm Charts, Kustomize or pure YAML/ JSON files.

This article involves:

KubeSphere DevOps was used to implement CI, and Argo CD was used to complete CD.
Argo CD continuously monitors changes of YAML files in a directory of Git repository and automatically deploits yamL files to K8s cluster.
Argo CD continuously monitors an image tag change in the Harbor image repository and automatically deploys the latest image to the K8s cluster.

Basic schematic diagram:

Prepare Git repository

Prepare two Git repositories, one source repository and one YAML file repository, with the source code separated from the YAML files.

The source repository can be referred to the following link. For offline reasons, the second example, Spring-Demo, is selected here:

Github.com/KubeSphere/…
Github.com/willzhang/s…

The YAML file repository can be referenced to the following link, named argocd-gitops:

Github.com/argoproj/ar…

Create a Javademo directory in the YAML repository and create two simple YAML files:

[root@jenkins git]# tree argocd-gitops/Argocd gitops / ├ ─ ─ javademo │ ├ ─ ─ javademo - deployment. Yaml │ └ ─ ─ javademo - SVC. YamlCopy the code

In the javademo-deployment.yaml example, the current image tag can be arbitrarily specified and replaced in real time when CI is executed:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: javademo
spec:
  replicas: 1
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: javademo
  template:
    metadata:
      labels:
        app: javademo
    spec:
      containers:
      - image: 10.39140.196.:8081/apps/javademo:replace
        name: javademo
        ports:
        - containerPort: 8080
Copy the code

javademo-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: javademo
spec:
  type: NodePort
  ports:
  - port: 8012
    targetPort: 8080
  selector:
    app: javademo
Copy the code

The deployment of Argo CD

Argo CD can be deployed in a variety of ways. Yaml files can be directly deployed.

kubectl create namespace Argo CD
kubectl apply -n Argo CD -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
Copy the code

If helm mode is used, you can specify the Argo CD Server service nodePort:

helm repo add argo https://argoproj.github.io/argo-helm

helm install Argo CD \
  --namespace=Argo CD --create-namespace \
  --set server.service.type=NodePort \
  argo/argo-cd
Copy the code

View the Pod in action:

[root@master ~]# kubectl -n Argo CD get pods
NAME                                             READY   STATUS    RESTARTS   AGE
argocd-application-controller-5db8c6f8f9-qnmtr   1/1     Running   0          8h
argocd-dex-server-84b5cbfbc9-fc7rf               1/1     Running   0          8h
argocd-redis-7c7c79dcd9-hjhgr                    1/1     Running   0          8h
argocd-repo-server-5fb9cbb945-9xmc7              1/1     Running   0          8h
argocd-server-8d8cb6488-pjwt4                    1/1     Running   0          8h
Copy the code

If KubeSphere is used to deploy the Argo CD, you need to configure the Helm repository of the Argo CD, enter the enterprise space, select an application template to upload the offline Helm Chart package, or configure the public Helm Repo address in the application repository.

After completion, enter the project, click Deploy new application, and select Argo CD Helm Chart to deploy:

Install the Argo CD CLI

To interact with the Argo CD API Server, we need to install the CLI command:

Cp argocd wget https://github.com/argoproj/argo-cd/releases/download/v1.7.10/argocd-linux-amd64 - Linux - amd64 / usr /local/bin/Argo CD
chmod +x /usr/local/bin/Argo CD

Argo CD version
Copy the code

If the Argo CD is deployed in YAML mode, change the serivce type to Nodeport to access the Argo CD API Server

kubectl patch svc argocd-server -n Argo CD -p '{"spec": {"type": "NodePort"}}'
Copy the code

Run the Argo CD Server Service command to query the nodeport information.

[root@master ~ ]# kubectl -n Argo CD get svcThe NAME TYPE CLUSTER - EXTERNAL IP - the IP PORT (S) AGE argocd dex - server ClusterIP 10.99.232.27 < none > 5556 / TCP, 5557 / TCP, 5558 / TCP 5d argocd-metrics ClusterIP 10.107.37.4 <none> 8082/TCP 5d Argocd -redis ClusterIP 10.106.160.6 <none> 6379/TCP 5d Argocd-repo-server ClusterIP 10.100.101.100 <none> 8081/TCP,8084/TCP 5d argocd-server NodePort 10.106.141.243 <none> 80:31195 / TCP, 443:32079 / TCP 5 d argocd - server - metrics ClusterIP 10.109.81.234 < none > 8083 / TCP 5 dCopy the code

Argo CD The default login user is admin, and the initial password is argocd-server Pod name. Obtain the Pod name

podName=`kubectl get pods -n Argo CD -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2`
Copy the code

Argo CD login 10.39.140.248:31195 --username admin --password $podName
Copy the code

Change the default password:

Argo CD account update-password \
  --current-password $podName \
  --new-password Argo CD@123
Copy the code

Argo CD UI:

https://10.39.140.248:31195
Copy the code

Deploy the Argo CD application

After logging in to Argo CD UI, select NEW APP to create application and select EDIT AS AYML:

Paste the following content and click CREATE on the upper left after SAVE. You can also use kubectl apply to execute the following content.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: javademo
  namespace: Argo CD
  finalizers:
    - resources-finalizer.Argo CD.argoproj.io
spec:
  project: default
  source:
    path: javademo
    repoURL: http://10.39.140.196:10080/gogs/argocd-gitops.git
    targetRevision: HEAD
  destination:
    namespace: apps
    server: https://kubernetes.default.svc
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
      allowEmpty: false
    syncOptions:
    - Validate=false
    - CreateNamespace=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m
Copy the code

Parameter Description:

Metadata: Specify the application name and specify Argo CD in the namespace. Add finalizers to delete K8S resources in cascading mode.
Source: specifies the URL of the Git repository where yamL files are located and the directory where yamL files are stored to be monitored. Argo CD will automatically deploy any changes to the files in this directory to the K8S cluster.
Destination field: specifies which K8S cluster and namespace to deploy the yamL file to be monitored.
SyncPolicy: Indicates the automatic synchronization policy and frequency. If this parameter is not specified, manual synchronization must be triggered.

If you are using a private Git repository, you need to create a certificate. The certificate in this case is the certificate to access the yamL file Git repository:

Equivalent Argo CD cli command:

Argo CD repo add http://10.39.140.196:10080/gogs/argocd-gitops - username gogs - password XXXXXXCopy the code

After creation, the Argo CD will automatically deploy yamL files in the Javademo directory of the Git repository to the K8s cluster. At this time, the application cannot start normally, because the image tag in the YAML file does not exist, and the image will fail to be pulled:

You can also use the Argo CD CLI to view deployed applications:

[root@master ~]# Argo CD app get javademo
Name:               javademo
Project:            default
Server:             https://kubernetes.default.svc
Namespace:          apps
URL:                https://10.39.140.248:31195/applications/javademo
Repo:               http://10.39.140.196:10080/gogs/argocd-gitops.git
Target:             HEAD
Path:               javademo
SyncWindow:         Sync Allowed
Sync Policy:        Automated (Prune)
Sync Status:        Synced to HEAD (1b96380)
Health Status:      Progressing

GROUP  KIND        NAMESPACE  NAME      STATUS  HEALTH       HOOK  MESSAGE
       Service     apps       javademo  Synced  Healthy            service/javademo unchanged
apps   Deployment  apps       javademo  Synced  Progressing        deployment.apps/javademo unchanged
Copy the code

Check Pod status in KubeSphere UI, keep trying to pull image:

Kubectl ImagePullBackOff:

[root@master ~]# kubectl -n apps get pods
NAME                       READY   STATUS             RESTARTS   AGE
javademo-64d46bff8-6dgjn   0/1     ImagePullBackOff   0          13m

[root@master ~]# kubectl -n apps get svcNAME TYPE cluster-ip external-ip PORT(S) AGE JavaDemo ClusterIP 10.111.56.180 < None > 8012/TCP 33MCopy the code

KubeSphere creates an assembly line

Create CI pipeline, use KubeSphere DevOps to complete source code compilation, image construction and push to Harbor warehouse, and finally update image field in YAML warehouse by Git commit.

Since the Argo CD is constantly monitoring changes to the YAML repository configuration file at this point, when the CI part does a Git push, the Argo CD is triggered to update the YAML file to the K8s cluster.

Create an empty pipeline under KubeSphere DevOps project, name it JavaDemo, go to the pipeline, select Edit Jenkinsfile, and copy the following:

pipeline {

    environment {
        GIT_URL='http://10.39.140.196:10080/gogs/spring-demo.git'
        GIT_CREDENTIAL_ID = 'git-id'
        GIT_BRANCH = 'master'
        REGISTRY = '10.39.140.196:8081 / apps/javademo'
        REGISTRY_CREDENTIAL_ID = 'harbor-id'
    }

    agent {
        node {
            label 'maven'
        }
    }

    stages {

        stage('SCM Checkout') {
            steps {
                git branch: "${GIT_BRANCH}".credentialsId: "${GIT_CREDENTIAL_ID}".url: "${GIT_URL}"
            }
        }

        stage('source build') {
            steps {
                container('maven') {
                    sh 'mvn clean package'

                }
            }
        }

        stage('docker build & push') {
            steps {
                script {
                    env.COMMIT_ID = sh(returnStdout: true.script: "git log -n 1 --pretty=format:'%h'").trim()
                    env.TIMESTRAP = sh(returnStdout: true.script: 'date +%Y%m%d%H%M%S').trim()
                    env.DOCKER_TAG = "dev_${TIMESTRAP}_${COMMIT_ID}_${BUILD_NUMBER}"
                }
                container('maven') {
                    withCredentials([usernamePassword(passwordVariable : 'DOCKER_PASSWORD' ,usernameVariable : 'DOCKER_USERNAME' ,credentialsId : "$REGISTRY_CREDENTIAL_ID" ,)]) {
                        sh 'docker build -t $REGISTRY:$DOCKER_TAG .'
                        sh 'echo "$DOCKER_PASSWORD" | docker login $REGISTRY -u "$DOCKER_USERNAME" --password-stdin'
                        sh 'docker push $REGISTRY:$DOCKER_TAG'
                    }
                }
            }
        }

        stage('update docker tag') {
            environment {
                BUILD_USER = 'admin'
                BUILD_USER_EMAIL = 'admin@Argo CD.com'
                YAML_REPO_URL='http://${username} : ${password} @ 10.39.140.196:10080 / gogs/argocd - gitops. Git'
            }

            steps {
                withCredentials([usernamePassword(passwordVariable : 'password' ,usernameVariable : 'username' ,credentialsId : "$GIT_CREDENTIAL_ID" ,)]) {
                    sh """ git config --global user.name "$BUILD_USER" git config --global user.email "$BUILD_USER_EMAIL" git clone ${YAML_REPO_URL} && cd argocd-gitops sed -i "s#$REGISTRY.*#${REGISTRY}:${DOCKER_TAG}#g" javademo/javademo-deployment.yaml git add -A && git commit -m "update tag: ${DOCKER_TAG}" && git push ${YAML_REPO_URL} """
                }
            }
        }
    }
}
Copy the code

Note that related parameters are modified. Two credentials are referenced in the pipeline:

GIT_CREDENTIAL_ID is the password of the gogs Git repository account on the Intranet
REGISTRY_CREDENTIAL_ID indicates the harbor warehouse account password

Before running the pipeline, relevant credentials need to be created in the DevOps project in advance and referenced in jenkinsfile later.

The final assembly line is as follows, click Run, wait for the completion of the assembly line, and check the status as success:

Looking at the pipeline build log, you can see that the following process was performed, including the final update docker tag step, which performed two key operations, sed replacing the mirror tag, and git push updating the YAML repository.

View images pushed to Harbor warehouse:

Argo CD detects yamL file changes and updates to K8s cluster:

Argo CD UI to view images used:

Argo CD synchronization can also be triggered by directly modifying the yaml file configuration in the Git repository, such as changing the service type to nodePort:

Wait for Argo CD to automatically synchronize configuration update to K8s cluster, and the browser accesses Java Web application in nodePort mode:

Deploy the Argo CD Image Updater

While the above example demonstrates the fact source for application deployment based on Git repository changes, the following example demonstrates another way of using mirrored tag changes as the fact source for application deployment. Argo CD provides the Argo CD Image Updater gadget to do this.

Argo CD Image Updater is a tool that automatically updates Kubernetes workload container images managed by Argo CD.

The tool is currently under development and has the following features and limitations:

Updates can only be managed by Argo CD and byHelmorKustomizeA mirror of the tool-generated application;
Default support for widely used container repositories: DockerHub, Harbor Private image repositories, etc.
Can use the matcher function to filter the label list returned by the mirror warehouse;
Mirror pull secrets must exist in the same Kubernetes cluster in which the Argo CD Image Updater is running (or accessible). It is currently not possible to obtain these secrets from another cluster.
In the current release, Argo CD Image Updater does not put any changes back into Git repositories.

Official documents:

argocd-image-updater.readthedocs.io/en/stable/

The deployment of Argo CD Image Updater is complicated. The following operations are performed:

1. Create a local user in Argo CD

To create an Argo CD image updater that requires credentials to access the Argo CD API Server, add the following user definitions to argoCd-CM using an account that image-updater has the appropriate API permissions:

# kubectl -n Argo CD edit cm argocd-cm
data:
  accounts.image-updater: apiKey
Copy the code

Create an access token for the user that copies the value of the token somewhere you will need it later.

Argo CD account generate-token --account image-updater --id image-updater
Copy the code

2. Grant RBAC permissions in Argo CD

Configure appropriate RBAC permissions for the image-updater user. Argo CD Image updater requires update and GET permissions for the application.

# kubectl -n Argo CD edit cm argocd-rbac-cm
data:
  policy.default: role:readonly
  policy.csv: | p, role:image-updater, applications, get, */*, allow p, role:image-updater, applications, update, */*, allow g, image-updater, role:image-updaterCopy the code

3. Install Argo CD Image Updater

Yaml file download: github.com/argoproj-la…

kubectl create ns argocd-image-updater
kubectl apply -n argocd-image-updater -f manifests/install.yaml
Copy the code

4. Configure the mirror vault

Even if you do not plan to use a private image repository, you need to configure at least one Empty registries.conf:

# kubectl -n argocd-image-updater edit cm argocd-image-updater-config
data:
  registries.conf: ""
Copy the code

Argocd-image-updater Pod will not start without this entry.

To use a private image warehouse, refer to the following configuration. The Harbor image warehouse is used as an example:

data:
  Argo CD.insecure: "true"
  log.level: debug
  registries.conf: | registries: - name: harbor api_url: http://10.39.140.196:8081 prefix: 10.39.140.196:8081 ping: yes insecure: yesCopy the code

Configure the API access token key

When installed from the inventory into the Kubernetes cluster, the Argo CD Image Updater will read the token needed to access the Argo CD API from the environment variable named Argo CD_TOKEN, The environment variable is set from argocd-image-upater -secret in the secret field named Argo cd. token.

The value of Argo cd.token should be set to the Base64 encoded value of the access token you generated above. As a shortcut, you can use Kubectl to generate the key and apply it to an existing resource:

YOUR_TOKEN=xxx
kubectl create secret generic argocd-image-updater-secret \
  --from-literal Argo CD.token=$YOUR_TOKEN --dry-run -o yaml |
  kubectl -n argocd-image-updater apply -f -
Copy the code

After the change, you must restart argocd-image-updater Pod, that is, run

kubectl -n argocd-image-updater rollout restart deployment argocd-image-updater
Copy the code

Create a new yamL repository Kustomize file

Since image updater only supports YAML of helm or Kustomize type, create a new yamL directory based on Kustomize and modify the parameters in YAML so that they do not conflict with the previous ones:

[root@jenkins git]# tree argocd-gitops/kustomize-javademo/├─ ├─ Class exercises, class exercises, class exercises, class exercises, class exercises, class exercises, class exercises, class exercises, class exercises, class exercisesCopy the code

javademo-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: javademo-tag
spec:
  replicas: 1
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: javademo-tag
  template:
    metadata:
      labels:
        app: javademo-tag
    spec:
      containers:
      - image: 10.39140.196.:8081/apps/javademo:replace
        name: javademo-tag
        ports:
        - containerPort: 8080
Copy the code

javademo-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: javademo-tag
spec:
  ports:
  - port: 8012
    targetPort: 8080
  selector:
    app: javademo-tag
Copy the code

kustomization.yaml

amePrefix: kustomize-

resources:
- javademo-deployment.yaml
- javademo-svc.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
Copy the code

Log in to the Argo CD UI and create a new Argo CD application. Add the annotations parameter to specify the address of the mirror to monitor, update the policy to latest, and modify the source path:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  annotations:
    argocd-image-updater.argoproj.io/image-list: Javademo = 10.39.140.196:8081 / apps/javademo
    argocd-image-updater.argoproj.io/javademo.update-strategy: latest
  name: javademo-tag
  namespace: Argo CD
  finalizers:
    - resources-finalizer.Argo CD.argoproj.io
spec:
  destination:
    namespace: apps
    server: https://kubernetes.default.svc
  project: default
  source:
    path: kustomize-javademo
    repoURL: http://10.39.140.196:10080/gogs/argocd-gitops.git
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
      allowEmpty: false
    syncOptions:
    - Validate=false
    - CreateNamespace=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m
Copy the code

KubeSphere UI to create a new CI pipeline, delete the update docker tag step, no longer need to trigger application deployment based on Git push:

pipeline {

    environment {
        GIT_URL='http://10.39.140.196:10080/gogs/spring-demo.git'
        GIT_CREDENTIAL_ID = 'git-id'
        GIT_BRANCH = 'master'
        REGISTRY = '10.39.140.196:8081 / apps/javademo'
        REGISTRY_CREDENTIAL_ID = 'harbor-id'
    }

    agent {
        node {
            label 'maven'
        }
    }

    stages {

        stage('SCM Checkout') {
            steps {
                git branch: "${GIT_BRANCH}".credentialsId: "${GIT_CREDENTIAL_ID}".url: "${GIT_URL}"
            }
        }

        stage('source build') {
            steps {
                container('maven') {
                    sh 'mvn clean package'

                }
            }
        }

        stage('docker build & push') {
            steps {
                script {
                    env.COMMIT_ID = sh(returnStdout: true.script: "git log -n 1 --pretty=format:'%h'").trim()
                    env.TIMESTRAP = sh(returnStdout: true.script: 'date +%Y%m%d%H%M%S').trim()
                    env.DOCKER_TAG = "dev_${TIMESTRAP}_${COMMIT_ID}_${BUILD_NUMBER}"
                }
                container('maven') {
                    withCredentials([usernamePassword(passwordVariable : 'DOCKER_PASSWORD' ,usernameVariable : 'DOCKER_USERNAME' ,credentialsId : "$REGISTRY_CREDENTIAL_ID" ,)]) {
                        sh 'docker build -t $REGISTRY:$DOCKER_TAG .'
                        sh 'echo "$DOCKER_PASSWORD" | docker login $REGISTRY -u "$DOCKER_USERNAME" --password-stdin'
                        sh 'docker push $REGISTRY:$DOCKER_TAG'
                    }
                }
            }
        }
    }
}
Copy the code

Check the pipeline log, and the image is successfully pushed to Harbor warehouse:

Harbor warehouse image tag update, Argo CD Image updater automatically updates the latest tag to the K8s cluster.

View the mirror tag:

Each time the Harbor repository generates a new image, Argo CD will automatically update it to the K8s cluster.

This article is published by OpenWrite!

Argo CD profile,

Prepare Git repository

The deployment of Argo CD

Install the Argo CD CLI

Deploy the Argo CD application

KubeSphere creates an assembly line

Deploy the Argo CD Image Updater

Related Posts

The end of the year is approaching, hand in hand to teach you to do a brilliant year-end summary report

Spring Boot open source tutorial 2019 will continue to update!

How is JSON data parsed, and in what ways?