When we try to understand how THE K8s cluster works, the Controller is definitely a difficulty. This is because there are a lot of controllers, and the implementation is very different; And the implementation of the controller uses some relatively obscure mechanisms, not easy to understand. However, we cannot bypass the controller because it is the “brain” of the cluster. In this article, the author analyzes the design process of a simple refrigerator to help readers understand the generation, function and implementation of the cluster controller.

K8s cluster core components

The following figure shows the core components of K8s cluster, including database ETCD, Scheduler Scheduler, cluster entrance API Server, Controller Controller, service proxy Kube-proxy and Kubelet that directly manages specific business containers.

These components can be logically divided into three parts:

  • Core components etc database;
  • API Server, an entry component for direct operations on ETCD;
  • Other components, “other components” here can be categorized because they can all be considered as controllers of the cluster.

Today we are going to talk about the principle of cluster controller.

Principle of controller

Although the controller is one of the more complex components in a K8s cluster, the controller itself is not new to us. We use washing machines, refrigerators, air conditioners and so on every day, all rely on the controller to work properly. In the controller principle section, we understand the principle of K8s cluster controller by thinking about the design process of a simple refrigerator.

Simple refrigerator

The refrigerator consists of five components: housing, cooling system, lighting system, thermostat and door.

A refrigerator only has two functions:

  • When someone opens the refrigerator door, the light in the refrigerator will automatically turn on;
  • When someone presses the thermostat, the cooling system adjusts the temperature inside the refrigerator according to the temperature setting.

Unified entrance

For the refrigerator above, we can simply abstract it into two parts: the unified operating entrance and all the components of the refrigerator.

Here, users can operate the fridge only through the entrance. This entry provides the user with two interfaces: opening and closing and adjusting the thermostat. When the user performs these two interfaces, the entry adjusts the state of the refrigerator door and the thermostat respectively.

However, there is a problem here, that is, users can neither turn on the lights inside the refrigerator nor adjust the temperature of the refrigerator through these two interfaces.

The controller

Controllers are created to solve the above problems. The controller is the bridge between the user’s actions and the correct state of the refrigerator’s components:

  • When the user opens the door, the controller observes the change of the door. It turns on the light in the refrigerator for the user.
  • When the user presses the temperature controller, the controller observes the temperature set by the user, it manages the refrigeration system for the user, and adjusts the temperature in the refrigerator.

Controller manager

A refrigerator has a lighting system and a cooling system, so it makes more sense for us to implement a separate controller for each component rather than one controller managing two components. At the same time, we implement a controller manager to maintain all these controllers in a unified way to ensure that these controllers work properly.

SharedInformer

The controller and controller manager above look pretty good already. But as the function of the refrigerator increases, many new controllers will be added. These controllers need to monitor the state changes of the components they care about through the entrance of the refrigerator. For example, the lighting controller needs to monitor the refrigerator door at all times. When a large number of controllers are constantly communicating with the inlet, the pressure on the inlet increases.

At this time, we will monitor the state changes of refrigerator components, a new module SharedInformer implementation.

SharedInformer, as the agent of the controller, monitors the state changes of refrigerator components for the controller, and notifies the corresponding controller of the state changes of different components according to the controller’s preference. Optimized, the SharedInformer can greatly relieve pressure on the refrigerator inlet.

ListWatcher

SharedInformer facilitates controller monitoring of refrigerator components, and the core function of this mechanism, of course, is to actively obtain component state and passively receive notification of component state change. Add these two features together and you get ListWatcher.

Assuming that the SharedInformer and refrigerator entry communicate over HTTP, HTTP Chunked Transfer Encoding is a good choice for ListWatcher. The controller sends a query to the refrigerator portal via ListWatcher and waits, and when the refrigerator component changes, the portal notifits controller via a segmented HTTP response. The controller sees the chunked response and thinks the response data hasn’t been sent yet, so it keeps waiting.

For example

From the evolution of a simple refrigerator, we have learned the meaning, role and realization of the controller. Now let’s go back to the K8s cluster. The K8s cluster implements a large number of controllers, and in the foreseeable future, new functional controllers will continue to appear, while some old controllers will be phased out. At present, we commonly use controllers, such as Pod controller, Deployment controller, Service controller, Replicaset controller, etc. Some of these controllers are implemented and managed by the Kube Controller Manager manager, while other controllers like route controller and Service controller are implemented by cloud Controller Manager. Cloud Controller Manager appears because in different cloud environments, the implementation of some controllers will be very different depending on cloud vendors and cloud environments. Such controllers are divided and implemented by cloud vendors based on cloud Controller Manager. Here, we take route controller and Service controller implemented by Ali Cloud K8s cluster Cloud Controller Manager as examples to briefly explain the working principle of K8s controller.

Service controller

First, the user asks the API Server to create a LoadBalancer service. The API Server receives the request and writes the details of this service to the ETCD database. This change is observed by the service controller. The service controller understands that a LoadBalancer service, in addition to including service records stored inside etCD, requires an SLB as the service entry point and several endpoints as the service back end. So the service controller requests SLB’s cloud OpenAPI and API Server to create SLB resources in the cloud and endpoints resources in the cluster.

Route controller

In the cluster Network chapter, we mentioned that when a node joins a K8s cluster, the cluster needs to add a route to the VPC routing table to set up the trunk road of the new node to the Pod network. And that’s what the route controller does. The routing controller does this in a very similar way to the service controller’s process above, which will not be described here.

conclusion

Basically, the controller of the K8s cluster acts as the brain of the cluster. With the controller, the K8s cluster has the opportunity to become an automatic, intelligent and useful system without being mechanical and passive.