Environment to prepare

  • OS: CentOS 7.5
  • Kubernetes v1.11.6
  • Etcd 3.3.10
  • Docker 1.13.1

What is a AdmissionWebhook

AdmissionWebhook is the Admission Controller of K8S. It is the admission controller of K8S. The Admission Controller is the gateway that intercepts (authenticated)API Server requests and can modify the request object or reject the request.

In short, it can be thought of as an interceptor, similar to middleware in a Web framework.

K8S provides many built-in Admission Controllers by default. You can view the admission Controller plugin supported by kube-apiserver startup command parameters.

[root @ node220] # kube - apiserver -- help | grep enable - admission - # plugins support plugin has the following AlwaysAdmit, AlwaysDeny, AlwaysPullImages, DefaultStorageClass, DefaultTolerationSeconds, DenyEscalatingExec, DenyExecOnPrivileged, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, Initializers, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodPreset, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, ValidatingAdmissionWebhook.Copy the code

Here the admission-plugins of enable are as follows

--enable-admission-plugins=PersistentVolumeClaimResize,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass ,ResourceQuota,NodeRestriction,ValidatingAdmissionWebhook,MutatingAdmissionWebhookCopy the code

I won’t go into detail about each plugin here, but you can find it online. Generally speaking, admission-plugins fall into three categories:

1. Mutating

2. Validate types

3. Both modifying and verifying types

These admission plugins form a chain of orders that determine who calls first, but do not affect usage.

There are two plugins of concern:

A, MutatingAdmissionWebhook ValidatingAdmissionWebhook

  • AdmissionWebhook MutatingAdmissionWebhook: modification operation
  • AdmissionWebhook ValidatingAdmissionWebhook: do verification operation

Reference kubernetes official blog a diagram to illustrate MutatingAdmissionWebhook and ValidatingAdmissionWebhook position:

Explain the process:

1. The API request reaches the K8S API Server

2. The request must be authenticated first

  • Kubectl calls require KubeconFig
  • Calling the K8S API directly requires a certificate +bearToken
  • Client-go calls also require KubeconFig

3. Perform a series of admission controller, including MutatingAdmissionWebhook and ValidatingAdmissionWebhook, serial execution MutatingAdmission Webhook list first

4. Verify the schema of the request object

5. Execute ValidatingAdmission’s Webhook list in parallel

6. Write etcd at last

Initializers vs AdmissionWebhooks

Both of them can dynamically and expandably load admission controller. Initializers are serialized execution. In high-concurrency scenarios, objects can easily stay in the uninitialized state, affecting the continued scheduling. Alpha Initializers have been removed in K8S version 1.14. See github.com/kubernetes/… Officially recommend AdmissionWebhook over Initializers; MutatingAdmissionWebhook is serial execution, ValidatingAdmissionWebhook is executed in parallel, the performance is better.

AdmissionWebhook application scenario

Istio is the application of AdmissionWebhook for automatic sidecar container injection. I’m currently using two application scenarios, and there are certainly others.

  • Automatic labeling

    For example, start an application, including Deployment, Service, ingress; How to quickly filter out which resources belong to applications? In K8S, POD, Service, and ingress are all independent resources. The fastest way to label these resources is to label them.

  • Automatically inject the Sidecar container

    How do I handle application monitoring and logs after an application is started? Injected into its pod with the help of the Sidecar container

The sidecar container for collecting application logs can be similar to the sidecar container for application monitoring as shown in the following figure

AdmissionWebhook demo

Enter actual combat stage, see demo

Demo address: github.com/yaoice/webh…

The functions are as follows:

  • The namespace for the admison-webhook-example =enabled tag is valid
  • Automatic labeling (POD, deplpoyment, service, and ingress can be automatically labeled with app.kubernetes.io/name=not_available)
  • Automatic Sidecar injection (POD automatically carries Nginx Sidecar Container)

Clone demo project

git clone https://github.com/yaoice/webhook-demo.git
Copy the code

Using scripts (istio team) to generate CertificateSigningRequest, generating secret (to the back of the webhook – use the API)

./deployment/webhook-create-signed-cert.sh 
Copy the code

Mutatingwebhook and ValidatingWebhook YAML are generated using scripts, and the variable CA_BUNDLE is automatically obtained from KubeconFig

cat ./deployment/validatingwebhook.yaml | ./deployment/webhook-patch-ca-bundle.sh > ./deployment/validatingwebhook-ca-bundle.yaml
cat ./deployment/mutatingwebhook.yaml | ./deployment/webhook-patch-ca-bundle.sh > ./deployment/mutatingwebhook-ca-bundle.yaml  
Copy the code

Compile the mirror

docker build --rm -t test/admission-webhook-example:v1 -f docker/Dockerfile .
Copy the code

Deploy webhook – API

kubectl apply -f ./deployment/mutatingwebhook-ca-bundle.yaml kubectl apply -f ./deployment/validatingwebhook-ca-bundle.yaml kubectl apply -f configmap.yaml kubectl apply -f deployment.yaml kubectl Yaml kubectl apply -f nginxconfigmap. Yaml #Copy the code

Assigning a label to the default namespace takes effect only for the namespaces of the admission-webhook-Example label

kubectl label namespace default admission-webhook-example=enabled
Copy the code

Deploy a BusyBox where sidecar is nginx

kubectl apply -f ./deployment/sleep.yaml
Copy the code

The POD is automatically labeled app.kubernetes.io/name. There are two containers in the POD

kubectl get pod sleep-5588cb5f94-5dl8f --show-labels 
NAME                     READY     STATUS    RESTARTS   AGE       LABELS
sleep-5588cb5f94-5dl8f   2/2       Running   0          27s       app.kubernetes.io/name=not_available,app=sleep,pod-template-hash=1144761950 
Copy the code

Service automatically tagged app.kubernetes.io/name

Kubectl get SVC sleep --show-labels NAME TYPE cluster-ip external-ip PORT(S) AGE labels sleep ClusterIP 10.68.4.5 < None >  80/TCP 4m app.kubernetes.io/name=not_availableCopy the code

Ingress is automatically tagged app.kubernetes.io/name

kubectl get ingresses.extensions sleep --show-labels 
NAME      HOSTS          ADDRESS   PORTS     AGE       LABELS
sleep     xx.sleep.com             80        4m        app.kubernetes.io/name=not_available
Copy the code

Development and debugging

If K8S is installed on a remote Server, how to debug it locally?

  1. Start the Webhook API in Goland, listening on local 6443

  1. Use SSH reverse proxy
SSH -r 6443:127.0.0.1:6443 root @ < IP k8s - master - >Copy the code

Listen on 6443 on K8S Server and redirect to local 6443

  1. Use K8S headless service to point to 6443 of K8S-master-IP
[root@node66 ~]# cat /data/deployment/service.yaml 
 apiVersion: v1
 kind: Service
 metadata:
   name: admission-webhook-example-svc
 spec:
   ports:
   - port: 443
     targetPort: 6443

 ---
 apiVersion: v1
 kind: Endpoints
 metadata:
   name: admission-webhook-example-svc
 subsets:
   - addresses:
       - ip: <k8s-master-ip>
     ports:
       - port: 6443
Copy the code

After the above steps, you can set breakpoints for debugging locally

Divergent thinking

AdmissionWebhook Multi-cluster application

If you have multiple K8S clusters, do you want to start a Webhook API in each cluster? Hence the following architecture, where all clusters share a Webhook API.

The Webhook configuration in cluster C1 and C2 points to A service in each cluster. This service is actually A headless service, which points to A service in Cluster A. NodePort also works), so all clusters share a Webhook API.

conclusion

AdmissionWebhook can like interceptors to intercept K8S API request, to modify the function with MutatingAdmissionWebhook, use ValidatingAdmissionWebhook validation functions.

Writing an AdmissionWebhook API

The request information can be obtained from the request field of AdmissionReview. Response is returned via the Response field setting of AdmissionReview

// AdmissionReview describes an admission review request/response.
 type AdmissionReview struct {
     metav1.TypeMeta `json:",inline"`
     // Request describes the attributes for the admission request.
     // +optional
     Request *AdmissionRequest `json:"request,omitempty" protobuf:"bytes,1,opt,name=request"`
     // Response describes the attributes for the admission response.
     // +optional
     Response *AdmissionResponse `json:"response,omitempty" protobuf:"bytes,2,opt,name=response"`
 }
Copy the code

2. Mutating is realized through JSON patch, and the corresponding structure is defined as follows:

 type patchOperation struct {
   Op    string      `json:"op"`
   Path  string      `json:"path"`
   Value interface{} `json:"value,omitempty"`
 }

 patches = append(patches, patchOperation{
   Op:    "add",
   Path: "/metadata/annotations",
   Value: true,
 })
Copy the code

Refer to the link

  • Github.com/morvencao/k…
  • Github.com/banzaicloud…
  • Banzaicloud.com/blog/k8s-ad…
  • Kubernetes. IO/blog / 2019/0…