Kubernetes cluster management, Kubefed Federation (v2), published in the earliest blog.ihypo.net/15716465002…

Kubefed (Federation V2) is a multi-cluster solution that the community is currently having a difficult time with. The current version is 0.1.0, and if you consider Federation V1, Kubefed is also a big one.

Why was Federation V1 deprecated

Federation V1 went into Beta with Kubernetes V1.6, but did not develop further until it was officially deprecated around Kubernetes V1.11. As for the reason that it was scrapped, the development team decided that cluster federation was more difficult than expected. There were many issues that v1 architecture did not take into account, such as:

  • The control plane components may fail, affecting the overall cluster efficiency.
  • Not compatible with new Kubernetes API resources.
  • Unable to effectively manage permissions across multiple clusters, such as without RBAC support.
  • The setup and strategy of the federated level relies on the Annotations content of API resources, which makes for poor elasticity.

From the perspective of Federation V1 architecture, Federation mainly consists of API Server, Controller Manager and external storage ETCD.

Federation API Server basically uses Kube API Server to provide a unified resource management portal, but only allows the use of Kubernetes resources supported by Adapter extension.

The Controller Manager coordinates the state between different clusters and manages all Kubernetes member clusters by communicating with the Api Server of the member clusters.

The overall Federation V1 architecture is similar to Kubernetes’ own architecture and manages the member cluster as a resource. But because V1 was not designed to flexibly add new Kubernetes resources and CRDS, new Adapters were created every time a new resource was created.

The original resource design was very inflexible, and the RBAC support problem made it impossible to manage the permissions of multi-cluster resources, thus aborting, and accumulated valuable lessons for V2.

Federation v2 design

Federation V2 uses CRD to implement the overall functionality, eliminating V1’s API Server by defining multiple custom resources (CRS), but thus introducing the concept of Host Cluster.

The basic concept

  • Federate: Federate means to join a set of Kubernetes clusters and provide them with a common cross-cluster deployment and access interface
  • KubeFed: Kubernetes Cluster Federation provides users with resource distribution, service discovery, and high availability across clusters
  • Host Cluster: Deploys the Kubefed API and allows the Kubefed Control Plane
  • Cluster Registration: Passedkubefedctl joinAdd member clusters to the Host Cluster
  • Member Cluster: a Cluster that is registered as a Member through the KubeFed API and managed by KubeFed. A Host Cluster can also serve as a Member Cluster.
  • ServiceDNSRecord: Records Kubernetes Service information and makes it accessible across the cluster through DNS
  • IngressDNSRecord: Records Kubernetes Ingress information and makes it accessible across the cluster through DNS
  • DNSEndpoint: a record (ServiceDNSRecord/IngressDNSRecord) the Endpoint information resources of the custom

architecture

Although Federation V2 has made major design changes and does away with the API Server, the overall architecture changes are not significant. Once the Federation Control Plan is deployed, you can see that it consists of two components:

$ kubectl -n kube-federation-system get deploy
NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
kubefed-admission-webhook    1/1     1            1            3s
kubefed-controller-manager   2/2     2            2            3s
Copy the code

Admission-webhook provides access control, and controller-manager handles custom resources and coordinates state between different clusters.

The working principle of

Logically, Federation V2 is divided into two parts: Configuration and Propagation.

The configuration design obviously draws lessons from v1 and configudes many contents that will change. Configuration mainly consists of two configurations:

  • Type Configuration: Describes the Type of resources that will be federally managed
  • Cluster Configuration: Saves API authentication information about federally managed clusters

Federation V2 takes great care of the Type Configuration and contains three key parts:

  • Templates is used to describe resources that are federated
  • Placement is used to describe the cluster that will be deployed
  • Overrides allows partial resource Overrides of partial cluster

The above has basically completed the definition of resources and provided resource description for Propagation. In addition, Federation V2 supports the definition of deployment policies and scheduling rules for finer management.

Get started

Kubefed: Kubernetes Federation (Kubefed)

reference

  • Github.com/kubernetes-…
  • www.kubernetes.org.cn/5702.html