K8s Intermediate – Installing a K8S cluster using RKE

preface

In order to quickly start learning k8S choreography, directly use the K8S cluster on Ali Cloud, this intermediate chapter here try to build k8S cluster from 0-1, and demonstrate some more classic cases: For example, use the self-built NFS server storage volume, automatic certificate issuing cert-manager let’s encrypt, Rancher2. x, and helm package management tool.

The environment

Three hosts are HUAWEI ECS, CentOS Linux release 7.6.1810 (Core)

Install the docker

Version 19.03.9

1. Uninstall the current Docker version (on demand)

yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine
Copy the code

2. Install the package management tool

   yum install -y yum-utils
   yum-config-manager \
       --add-repo \
       https://download.docker.com/linux/centos/docker-ce.repo
   Copy the code

3. Check the version

   yum list docker-ce --showduplicates | sort -r
   Copy the code

4. Install Docker – specify the version

   Yum install docker-ce-19.03.9-3.el7 docker-ce-cli-19.03.9-3.el7 containerd. IOCopy the code

5. Start the docker

   systemctl start docker
   Copy the code

6. Modify the configuration

   {
     "registry-mirrors": [
       "https://3p42xjxk.mirror.aliyuncs.com"."https://registry.docker-cn.com"."http://hub-mirror.c.163.com"."https://docker.mirrors.ustc.edu.cn"]."log-driver": "json-file"."log-opts": {
       "max-size": "100m"}}Copy the code

• Registry -mirrors: accelerates mirroring
  • Log-driver: indicates the log engine
  • Log-opts: log configuration

7. Restart the docker

   systemctl restart docker
   Copy the code

Install Kubectl

• Source configuration

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
Copy the code

• The installation

[root@mldong ~]# yum install -y kubectl
Copy the code

Preparations before installing rKE

1. Disable Swap on all Woker nodes (Swap)

   swapoff -a
   Copy the code

2. Check if the following modules exist – all nodes

   for module in br_netfilter ip6_udp_tunnel ip_set ip_set_hash_ip ip_set_hash_net iptable_filter iptable_nat iptable_mangle iptable_raw nf_conntrack_netlink nf_conntrack nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat nf_nat_ipv4 nf_nat_masquerade_ipv4 nfnetlink udp_tunnel veth vxlan x_tables xt_addrtype xt_conntrack xt_comment xt_mark xt_multiport  xt_nat xt_recent xt_set xt_statistic xt_tcpudp; do if ! lsmod | grep -q $module; then echo "module $module is not present"; fi; doneCopy the code

3. Modify sySCtl configuration – All nodes

   vi /etc/sysctl.conf
   ## add as follows
   net.bridge.bridge-nf-call-iptables=1
   #Reload the configuration
   sysctl -p /etc/sysctl.conf
   Copy the code

4. Create users – All nodes

   #Create a user
   useradd rkeuser
   #Set password for user
   passwd rkeuser
   Copy the code

5. Add a user to a Docker group

   usermod -aG docker rkeuser
   Copy the code

6. Configure cryptographic-exempt login on the master node

   #Generate a public-private key pair
   ssh-keygen  -t rsa -C '[email protected]'
   #Copy the master node's public key to all nodes (including itself) - making the newly created rkeuser userSsh-copy-id [email protected] ssh-copy-id [email protected] ssh-copy-id [email protected]Copy the code

7. Verify that rkeuser has docker command permissions

   #The loginSSH [email protected]#Run the docker command
   docker ps
   Copy the code

8. Port requirements

   Open 6443-KubeAPI and 2379-etcd

9. SSH server configuration

   vi /etc/ssh/sshd_config
   
   #TCP forwarding is allowed
   AllowTcpForwarding yes
   Copy the code

The installation is introduced

1. Download rKE binary package -master

   https://github.com/rancher/rke/releases

   Wget HTTP: / / https://github.com/rancher/rke/releases/download/v1.2.4-rc9/rke_linux-amd64Copy the code

2. Modify the file name and perform run permissions

   mv rke_linux-amd64 /usr/local/bin/rke
   chmod +x /usr/local/bin/rke
   Copy the code

3. Viewing the Version Number

   [root@mldong01 download]# rke --version
   rke version v1.2.4-rc9
   Copy the code

4. Use RKE to generate configuration files

   rke config --name cluster.yml
   Copy the code

   You can also use the sample file below

   nodes:
   - address: 192.168. 0245.
     port: "22"
     internal_address: 192.168. 0245.
     role:
     - controlplane
     - worker
     - etcd
     hostname_override: "mldong01"
     user: rkeuser
     ssh_key_path: ~/.ssh/id_rsa
     ssh_agent_auth: true
     labels: {}
     taints: []
   - address: 192.168. 054.
     port: "22"
     internal_address: 192.168. 054.
     role:
     - worker
     hostname_override: "mldong02"
     user: rkeuser
     ssh_key_path: ~/.ssh/id_rsa
     ssh_agent_auth: true
     labels: {}
     taints: []
   - address: 192.168. 022.
     port: "22"
     internal_address: 192.168. 022.
     role:
     - worker
     hostname_override: "mldong03"
     user: rkeuser
     ssh_key_path: ~/.ssh/id_rsa
     ssh_agent_auth: true
     labels: {}
     taints: []
   kubernetes_version: "V1.19.6 - rancher1-1"
   cluster_name: "mldong-k8s"
   Copy the code

   Main parameters:

   Nodes []. Address: indicates the external IP address

   Nodes []. Port: indicates the SSH port number

   Nodes []. Internal_address: indicates the internal IP address

   Nodes []. Role: node role, array, three options [controlplane,worker,etcd]

   Nodes []. Hostname_override: indicates the virtual domain name

   Nodes []. User: indicates the SSH user name

   Nodes []. Ssh_key_path: indicates the SSH private key

   Nodes []. Ssh_agent_auth: Enables SSH authentication

   Kubernetes_version: k8S version. You can run the rke config –list-version –all command to view the supported versions

   Cluster_name: indicates the cluster name

5. Start installation

   rke up --config cluster.yml
   Copy the code

   

   The installation process requires downloading an image, which takes a long time

   

6. After the installation is successful, related files are generated

• cluster.yml: configuration file of the RKE cluster.
• kube_config_cluster.yml: of the clusterContains the authentication credentials to obtain all permissions for the cluster.
• cluster.rkestate:Kubernetes Cluster status fileContains the authentication credentials to obtain all permissions for the cluster. This file is created when using RKE V0.2.0.

#! /bin/bash
#Kill all running containers
docker stop $(docker ps -a -q)

#Delete all containers
docker rm -f $(docker ps -qa)

#Delete all container volumes
  docker volume rm $(docker volume ls -q)
  
#Unmount directory
  for mount in $(mount | grep tmpfs | grep '/var/lib/kubelet' | awk '{ print $3 }') /var/lib/kubelet /var/lib/rancher; do umount $mount; done
  
#Deleting a Residual Path
rm -rf /etc/ceph \
       /etc/cni \
       /etc/kubernetes \
       /opt/cni \
       /opt/rke \
       /run/secrets/kubernetes.io \
       /run/calico \
       /run/flannel \
       /var/lib/calico \
       /var/lib/etcd \
       /var/lib/cni \
       /var/lib/kubelet \
       /var/lib/rancher/rke/log \
       /var/log/containers \
       /var/log/pods \

#Clearing network Interfacesnetwork_interface=`ls /sys/class/net` for net_inter in $network_interface; do if ! echo $net_inter | grep -qiE 'lo|docker0|eth*|ens*'; then ip link delete $net_inter fi done  
#Residue cleaning processport_list='80 443 6443 2376 2379 2380 8472 9099 10250 10254' for port in $port_list do pid=`netstat -atlnup|grep $port |awk '{print $7}'|awk -F '/' '{print $1}'|grep -v -|sort -rnk2|uniq` if [[ -n $pid ]]; then kill -9 $pid fi done pro_pid=`ps -ef |grep -v grep |grep kube|awk '{print $2}'` if [[ -n $pro_pid ]]; then kill -9 $pro_pid fi#Clearing Routing Rules 
sudo iptables --flush
sudo iptables --flush --table nat
sudo iptables --flush --table filter
sudo iptables --table nat --delete-chain
sudo iptables --table filter --delete-chain
#Restart the docker
sudo systemctl restart docker
Copy the code