Background:
Centos8 + Kubeadm1.20.5 + Cilium + HUBble Environment Upgrade to 1.21: Upgrade to 1.21.0 Boring to check the cluster today suddenly found a problem:
[root@sh-master-01 ~]# kubectl get pods -n default -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE Readiness Gates CSI-APP 1/1 Running 11 106D 10.0.4.204 SH-Work-01 < None > < None > NGINX 1/1 Running 0 13D 10.0.4.60 Sh-work-01 <none> <none> nginx-1-kkfvd 1/1 Running 0 13d 10.0.5.223 sh-work-02 <none> <none> nginx-1-klgpx 1/1 Running 0 13D 10.0.4.163sh-work-01 < None > < None > nginx-1-s5mzp 1/1 Running 0 13D 10.0.3.163sh-work-03 < None > < None > Nginx-2-8CB2J 1/1 Running 0 13D 10.0.3.228sh-work03 <none> <none> nginx-2-L527J 1/1 Running 0 13D 10.0.5.245 Sh-work-02 <none> <none> nginx-2-qnsrq 1/1 Running 0 13d 10.0.4.77 sh-work-01 <none> <none> php-apache-5b95f8f674-clzn5 Running 316D 10.0.5.252 sh-work-02 < None > < None > Running 316D 10.0.3.64 sh-work-02 < None > Running 316D 10.0.5.252 sh-work-02 < None > Pod-nodeaffinity 1/1 Running 0 13d 10.0.4.118 sh-work-01 <none> <none> pod-prefer 1/1 Running 0 13d 10.0.5.181 Sh - work - 02 < none > < none > pod - 1/1 prefer1 Running 0 13 d 10.0.3.54 sh - work - 03 < none > < none > with 0/1 - node - affinity ImagePullbackoff 0 13D 10.0.4.126 sh-work-01 < None > < None > with-pod-affinity 0/1 imagePullbackoff 0 13D 10.0.5.30 Sh-work-02 < None > < None > with-pod-antiaffinity 1/1 Running 0 13D 10.0.4.159 sh-work-01 < None > < None >I remember my ServiceSubnet: 172.254.0.0/16 and PodSubnet: 172.3.0.0/16.
Take another look at the network status of the service:
[root@sh-master-01 ~]# kubectl get svc -n default -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 172.254.0.1 <none> 443/TCP 106d <none>
php-apache ClusterIP 172.254.13.205 <none> 80/TCP 99d run=php-apacheThis IP section is correct! What’s the matter?
Review the problem and solve it
1. Identify the problem
Cilium is installed Oprator way, find the pod of Oprator:
kubectl get pods -n kube-system -o wide
To view the log output:
kubectl logs -f cilium-operator-f595b8f7d-7rvzz -n kube-systemThe ipv4CIDRs is 10.0.0.0/8!
Read the official documentation
To build a cluster, I refer to Fat Zhai Shuangwen’s blog:02-02. Build a Kubernetes cluster by Kubeadm + Cilium. The more complete Cilium cluster setup on the blog looks like we wrote it:Centos8 + Kubeadm1.20.5 + Cilium + Hubble. The initial suspicion is that Helm installed Cilium without configuring the network components in config.yaml. Take a closer look at the official GitHub repository file configuration:https://github.com/cilium/cilium/tree/v1.10.2/install/kubernetes/cilium.
Note: my version is 1.9.7, normal to find their corresponding version of the configuration file to see it. I didn’t have to switch branches to see…
Contrast the official document: at the time of helm installation does not specify ipam. Operator. ClusterPoolIPv4PodCIDR parameters! Many articles on the Internet are also rushed, and do not specify the IPv4 CIDR in detail that the POD network is the default 10.0.0.0/8 CIDR! My CVM VPC network default is 10.0.0.0/8 large network, so there should be IP address conflict!
3. Find the right reference
Google search documents found an Amazon blog:https://aws.amazon.com/cn/blogs/containers/a-multi-cluster-shared-services-architecture-with-amazon-eks-using-cilium-clu stermesh/
Amazon’s blog is still very good. The installation process is very detailed can refer to:
As for my cluster can only be upgraded…
HELM Upgrade Cilium Cilium/Cilium -- Version 1.9.7 -- Namespace = Kube-system --set Ipam. Operator. ClusterPoolIPv4PodCIDR = "172.3.0.0/16"Visual inspection should be such a special emphasis, copy the installation of others or to have a look at the official document of the detailed parameter definition! Most of these cilium files available in China don’t have podCIDR configuration, do they? Reference other people’s blog at the same time must remember to carefully look at the official document parameters!