This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money.

1 overview

Node and Pod affinity is the ability to attract a Pod to a set of nodes (as a preference or mandatory requirement).

Taints, by contrast, allow a node to reject a group of pods.

Tolerations is applied to pods, allowing (but not mandatory) pods to schedule to nodes with matching stains.

Taints and Tolerations work together to ensure that pods aren’t scheduled to inappropriate nodes. One or more smudges are applied to nodes; This indicates that the node should not accept any Pod that does not tolerate stains.

Note: We found that Pod will not be scheduled to the master node of K8S in ordinary use, because there is a stain on the master node.

2 Taints the stain

2.1 Composition of Taints

The kubectl taint command can be used to taint a Node. Once taint is placed on a Node, it has a mutually exclusive relationship with Pod, allowing Node to reject Pod scheduling, or even expel existing pods from Node.

Each stain consists of the following:

key=value:effect
Copy the code

Each stain has a key and value as a label for the stain, and effect describes the effect of the stain. Taint Effect currently supports the following options:

  • NoSchedule: indicates that K8S will not dispatch pods to nodes with this stain
  • PreferNoSchedule: indicates that K8S will try to avoid scheduling pods to nodes with this stain
  • NoExecute: indicates that K8S will not dispatch Pod to the Node with the stain and will expel existing Pod from the Node

2.2 NoExecute Note

Taint effect value NoExecute, which affects pods already running on the node:

  • If a Pod cannot tolerate a TAINt with effect value NoExecute, the Pod is immediately ejected
  • If Pod tolerates taint with an effect value of NoExecute, and tolerationSeconds are not specified in the Toleration definition, Pod will always run on this node.
  • If Pod tolerates taint with an effect value of NoExecute, but tolerationSeconds are specified in the Toleration definition, that represents the length of time Pod can continue to run at that node.

2.3 Taints stain setting

Taints (Taints) view

K8s Master node View

kubectl describe node k8s-master
Copy the code

Taints added

[root@centos ~]$kubectl taint node 10.1.2.8 disk= SSD :NoSchedule node/10.1.2.8 taintedCopy the code

A taint was added on node 10.1.2.8 with key as disk, value as SSD, and effect as NoSchedule. This means that no Pod can be scheduled to 10.1.2.8 unless there is a matching tolerance.

Taints removed

[root@centos ~]$kubectl taint nodes 10.1.2.8 disk= SSD: noschedule-node /10.1.2.8 untainted [root@centos ~]$kubectl Describe node 10.1.2.8 Name: 10.1.2.8 Roles: < None > Labels: beta.kubernetes.io/arch=amd64 beta.kubernetes.io/instance-type=S2.LARGE8 beta.kubernetes.io/os=linux cloud.tencent.com/node-instance-id=ins-e9d2sh1i failure-domain.beta.kubernetes.io/region=gz Failure - domain. Beta. Kubernetes. IO/zone = 100002 kubernetes. IO/arch = amd64 kubernetes. IO/hostname = 10.1.2.8 kubernetes.io/os=linux node.kubernetes.io/instance-type=S2.LARGE8 topology.kubernetes.io/region=gz topology.kubernetes.io/zone=100002 Annotations: csi.volume.kubernetes.io/nodeid: {" com. Tencent. Cloud. Csi. CFS ":" 10.1.2.8} "node. The alpha. Kubernetes. IO/TTL: 0 volumes.kubernetes.io/controller-managed-attach-detach: true CreationTimestamp: Sun, 03 Oct 2021 08:36:20 +0800 Taints: < None > ## Stain removedCopy the code

3 Tolerations tolerance

A Node with a taint will have a mutually exclusive relationship between NoSchedule, PreferNoSchedule, NoExecute, and Pod based on taint’s effect, and Pod will not be scheduled to Node to some extent.

But Tolerations can be set up on pods, which means that pods with Tolerations will tolerate stains and can be scheduled to nodes with stains.

Pod. Spec. Tolerations example

tolerations:
- key: "key"
  operator: "Equal"
  value: "value"
  effect: "NoSchedule"
---
tolerations:
- key: "key"
  operator: "Exists"
  effect: "NoSchedule"
---
tolerations:
- key: "key"
  operator: "Equal"
  value: "value"
  effect: "NoExecute"
  tolerationSeconds: 3600
Copy the code

Important Note:

  • Key, value, and effect must be the same as taint on Node
  • The operator has a value ofExistsValue is ignored. As long as you have a key and an effect
  • TolerationSeconds: Pod tolerates taint with NoExecute; When tolerationSeconds is specified, it represents the length of time that pod can continue to run on the node.

3.1 When the key is not specified

If the key and effect values are not specified and operator Exists, all stains are tolerated.

tolerations:
- operator: "Exists"
Copy the code

3.2 When the effect value is not specified

When no effect value is specified, all effects of the smudge key can be matched

tolerations:
- key: "key"
  operator: "Exists"
Copy the code

3.3 When Multiple Masters exist

When multiple masters exist, you can set the following parameters to prevent resource waste:

kubectl taint nodes Node-name node-role.kubernetes.io/master=:PreferNoSchedule
Copy the code

What about multiple Taints and Tolerations

You can have Taints on the same node and Tolerations on the same pod. Kubernetes handles multiple stains and tolerations like a filter: it starts with all the stains on the node and ignores those that can be matched by Pod tolerations; Retain the remaining non-negligible stains, and the stain effect has a display effect on Pod: in particular:

  • If there is at least one non-ignorable stain, effect is NoSchedule, then Kubernetes will not schedule pods to that node
  • If there is no non-ignorable stain with effect as NoSchedule, but there is at least one non-ignorable stain with effect as PreferNoSchedule, then Kubernetes will try not to schedule pods to that node
  • If there is at least one non-ignorable stain, effect is NoExecute, then the Pod is expelled from the node (if it is already running on the node) and will not be scheduled to the node (if it is not already running on the node).

\