This article is the first part of Kubernetes series. It will introduce Docker and Kubernetes two popular open source products. The main content includes: basic concepts, basic components, Kubernetes architecture.

The basic concept

What is the Docker

Docker was an internal project initiated by dotCloud founder Solomon Hykes in France. Docker is an innovation based on dotCloud’s cloud service technology over the years. In March 2013, it opened source with Apache 2.0 license. The main code of its project is maintained on GitHub. Since Docker opened source, it has been widely discussed and concerned.

Docker uses the Go language launched by Google for development and implementation. The process encapsulation and isolation is based on cgroup and Namespace of Linux kernel and Union FS of AUFS class, which belongs to the virtualization technology at the operating system level. Because a quarantined process is independent of the host and other quarantined processes, it is also called a container (more on the concept of “container” later). Docker is further packaged on the basis of containers, from network interconnection, file system to process isolation, which greatly simplifies the creation and maintenance of containers and makes Docker technology more portable and fast than virtual machine technology.

Here are two images of how Docker differs from traditional virtualization. Docker container application process directly runs on the host kernel, the container does not have its own kernel, there is no hardware virtualization; However, the traditional virtual machine technology is to create a set of virtual hardware, run a complete operating system on it, and then run the required application processes on the system. Therefore, containers are much lighter than traditional virtual machines.

Traditional virtualization

Docker

Why Docker?

Docker is an emerging virtualization way, which has many advantages over the traditional virtualization way.

More efficient utilization of system resources

Because containers do not require additional overhead such as hardware virtualization and running a full operating system, Docker has a higher utilization of system resources.

Faster startup time

Because Docker container applications run directly in the host kernel, there is no need to start the complete operating system, so they can be started in seconds or even milliseconds. Greatly saving development, testing, deployment time.

Operating environment consistency

A common problem in development is environmental consistency. Because the development environment, test environment, and production environment are inconsistent, some bugs are not found in the development process. The image of Docker provides a complete runtime environment in addition to the kernel, ensuring the consistency of the application running environment, so that there will no longer be “no problem with this code on my machine”.

Continuous delivery and deployment

The most desirable thing for development and operations is a single creation or configuration that can run anywhere. With Docker, continuous integration, continuous delivery and deployment can be achieved by customizing application images. Developers can use Dockerfile for image building and Integration testing with Continuous Integration systems, while operations can quickly deploy the image directly in a variety of environments. Even automatic Deployment with Continuous Delivery/Deployment systems.

Moreover, using Dockerfile to make image construction transparent, not only the development team can understand the application environment, but also facilitate the operation and maintenance team to understand the conditions required for application operation, which helps to better deploy the image in the production environment.

Easier migration

Docker ensures the consistency of execution environment, making application migration easier. Docker can run on many platforms, whether physical machine, virtual machine, public cloud, private cloud, or even laptop, and its running results are consistent. Therefore, users can easily migrate an application running on one platform to another without worrying that the application will not run properly due to the change of the operating environment.

Easier maintenance and extension

Docker uses layered storage and image technology, which makes it easier to reuse the repeated parts of the application, and also makes it very easy to maintain and update the application and further expand the image based on the basic image. In addition, Docker team maintains a large number of high-quality official images together with various open source project teams, which can be used directly in the production environment and can be further customized as a basis, greatly reducing the cost of image production of application services.

Docker images and containers

The slogan of Docker is “Build, Ship and Run Any App, Anywhere.” It basically means that after compiling an application, it can Run Anywhere. Unlike traditional programs, once changing the running environment, there will often be the problem of missing this library and that package. So how does Docker do this?

In short, it builds an image of everything the application depends on when it is compiled (sort of like static compilation of a program — only like that). We call this compiled object a Docker Image, and when Docker deamon (Docker daemon/server) runs the Image, we call it a Docker Container. It is easy to understand that the relationship between Docker image and Docker container is just like the relationship between program and process (of course, the essence is different).

Images and the Layers

Each Docker Image refers to several read-only layers, which vary from file system to file system. These layers stack together to form the root filesystem of the Container. The image below is an image of Ubuntu 15.04, consisting of four image layers:

The Container and the Layers

The main difference between a container and an image is the writable layer at the top (the “Container Layer”). Everything that the container does at runtime is written to this writable layer, and when the container is deleted, the writable layer is deleted, but the underlying image remains the same. So, different containers have their own writable layer, but can share the same underlying image. The following figure shows multiple containers sharing the same Ubuntu 15.04 image.

Docker storage driver is responsible for managing the read-only image layer and writable container layer. Of course, different drivers implement different ways, but there are two key technologies behind: Stackable image layer and copy-on-write (CoW) technology.

Docker data persistence

At first, Docker was generally only used in stateless computing scenarios. However, with the development of Docker, data volume technology can also achieve data persistence. A Data volume is a directory on a host that is mounted to a container. This Data volume is not controlled by the storage driver. All operations on this Data volume bypass the storage driver and its performance is limited only by the local host. In addition, we can mount any number of data volumes into containers, and different containers can share the same data volume.

The following figure shows a Docker host running two containers. Each container has its own address space on the host (/var/lib/docker-/…). In addition, they share the same /data directory on the host.

Reference Documents:

Yeasy. Gitbooks. IO/docker_prac… Yeasy. Gitbooks. IO/docker_prac… Docs.docker.com/storage/sto…

Introduction of Kubernetes

Kubernetes is Google’s open source container cluster management system. It is an open source version of Borg, Google’s large-scale container management technology.

  • Container-based application deployment, maintenance, and rolling upgrade
  • Load balancing and service discovery
  • Cluster scheduling across machines and across regions
  • Automatic telescopic
  • Stateless and stateful services
  • Extensive Volume support
  • The plug-in mechanism ensures extensibility

Kubernetes is growing very quickly and has become a leader in container choreography.

What is Kubernetes

Kubernetes offers a number of features that simplify application workflow and speed up development. In general, a successful application choreography system requires a high degree of automation, which is why Kubernetes was designed as an ecosystem platform for building components and tools that make it easier to deploy, scale, and manage applications.

Users can use labels to organize and manage resources in their own way, and annotations can be used to customize descriptions of resources, such as providing status checks for management tools.

In addition, the Kubernetes controller is built on the same API that developers and users use. Users can write their own controllers and schedulers, and can extend the functions of the system through various plug-in mechanisms. This design makes it easy for users to build applications on top of Kubernetes.

What is Kubernetes not

Kubernetes is not a platform-as-a-service system in the traditional sense. It reserves the freedom of choice for the user.

  • Without limiting the types of applications supported, it does not interfere with application frameworks, nor does it limit the supported languages (such as Java, Python, Ruby, etc.), Kubernetes is designed to support an extremely diverse range of workloads, including stateless, statically, and data-processing workloads. As long as the application can run in the container, it will run fine on Kubernetes.
  • It does not provide built-in middleware (such as message-oriented middleware), data processing frameworks (such as Spark), databases (such as mysql), or clustered storage systems (such as Ceph). These applications run directly on Top of Kubernetes.
  • There is no market for click-and-deploy services.
  • The code is not deployed directly and the user’s application is not built, but the user can build the continuous integration (CI) workflow required on top of Kubernetes.
  • Allows users to select their own logging, monitoring, and alarm systems.
  • No application configuration language or system (such as JsonNet) is provided.
  • No machine configuration, maintenance, management or self-healing system is provided.

In addition, there are already many PaaS systems running on Kubernetes, such as Openshift, Deis and Eldarion. You can also build your own PaaS system, or just use Kubernetes to manage your container applications.

Of course, Kubernetes is more than just a “choreography system,” it eliminates the need for choreography. Kubernetes uses declarative apis and a set of separate, composable controllers to ensure that the application is always in the desired state, and users do not need to care about how the intermediate state is changed. This makes the whole system easier to use and more powerful, reliable, resilient and scalable.

Reference Documents:

Kubernetes. IO/docs/concep…

Based on the component

Core components

Kubernetes consists of the following core components:

  • Etcd: saves the status of the entire cluster.
  • Apiserver: provides a unique entry point for resource operation and provides authentication, authorization, access control, API registration, and discovery mechanisms.
  • Controller Manager: Maintains cluster status, such as fault detection, automatic expansion, and rolling update.
  • Scheduler: Responsible for resource scheduling, scheduling Pod to corresponding machine according to the scheduled scheduling policy;
  • Kubelet: Responsible for maintaining the container lifecycle, as well as managing Volume (CVI) and network (CNI);
  • Container Runtime: Manages images and actually runs pods and containers (CRI).
  • Kube-proxy: provides intra-cluster Service discovery and load balancing for services

In addition to the core components, there are some recommended Add-ons:

  • Kube-dns: provides DNS services for the entire cluster
  • Ingress Controller: Provides an extranet entry for services
  • Heapster: Provides resource monitoring
  • Dashboard: Provides the GUI
  • Federation: Provides clustering across availability zones
  • Fluentd-elasticsearch: collects, stores, and queries cluster logs

Component Details

Etcd

Etcd is a distributed key-value store developed by CoreOS based on Raft for service discovery, shared configuration, and consistency assurance (database master selection, distributed locks, etc.).

Etcd main functions:

  • Basic key-value storage
  • Monitoring mechanism
  • Key expiration and renewal mechanisms for monitoring and service discovery
  • Atomic CAS and CAD for distributed locks and leader elections

kube-apiserver

Kube-apiserver is one of the most important core components of Kubernetes. It provides the following functions:

  • Provides REST apis for cluster management, including authentication and authorization, data verification, and cluster status change
  • Provide a hub for data interaction and communication between other modules (other modules query or modify data through the API Server, only the API Server can directly operate ETCD)

kube-controller-manager

Controller Manager consists of Kube-Controller-Manager and Cloud-Controller-Manager. It is the brain of Kubernetes. It monitors the state of the whole cluster through apiserver. And ensure that the cluster is working as expected.

The kube-controller-Manager consists of a series of controllers

  • Replication Controller
  • Node Controller
  • CronJob Controller
  • Daemon Controller
  • Deployment Controller
  • Endpoint Controller
  • Garbage Collector
  • Namespace Controller
  • Job Controller
  • Pod AutoScaler
  • RelicaSet
  • Service Controller
  • ServiceAccount Controller
  • StatefulSet Controller
  • Volume Controller
  • Resource quota Controller

cloud-controller-manager

It is required when Kubernetes enables the Cloud Provider, which is used to coordinate with the control of the Cloud service Provider. It also includes a series of controllers, such as:

  • Node Controller
  • Route Controller
  • Service Controller

kube-scheduler

The KuBE-Scheduler assigns scheduling pods to nodes in the cluster. It listens to the Kube-Apiserver, looks for pods that have not yet been assigned nodes, and then assigns nodes to these pods according to the scheduling policy (update the NodeName field of the Pod).

The scheduler needs to take into account a number of factors:

  • The fair scheduling
  • Efficient utilization of resources
  • QoS
  • Affinity and anti – affinity
  • Data Localization
  • Inter-workload Interference
  • deadlines

Kubelet

A kubelet server process runs on each node. By default, it listens on port 10250, receives and executes instructions from the master, and manages pods and containers in pods. Each Kubelet process registers node information on API Server, reports node resource usage to master node regularly, and monitors node and container resources through cAdvisor.

Container runtime

One of the most important components of Kubernetes is the Container Runtime, which really manages the image and Container life cycle. Kubelet interacts with the Container Runtime through the Container Runtime Interface (CRI) to manage images and containers.

kube-proxy

Each machine runs a Kube-proxy service that listens for service and endpoint changes in THE API server and configures load balancing (TCP and UDP only) for the service using iptables, etc.

Kube-proxy can run directly on the physical machine or as static POD or daemonset.

Kube-proxy currently supports several implementations:

  • Userspace: the earliest load-balancing scheme that listens on a port in userspace to which all services are forwarded via iptables and then load balanced internally to the actual Pod. The main problem of this method is low efficiency and obvious performance bottleneck.
  • Iptables: the recommended solution implements service load balancing based on iptables rules. The main problem of this method is that too many iptables rules are generated when there are too many services. Non-incremental update will introduce certain delay, resulting in significant performance problems in large-scale cases
  • Ipvs: To address the performance problems of the Iptables mode, V1.8 added the IPVS mode in an incremental manner to ensure that the connection is kept open during service updates
  • Winuserspace: Same as userspace, but only works on Windows.

Kubernetes architecture

The K8s setup consists of several parts, some of which are optional and some of which are required for the entire system to run. Below is the global architecture diagram of K8S

The Master has three components: API Server, Scheduler, and Controller. API Server provides friendly and easy to use API for external calls, while there are many powerful tools to make API calls more simple, such as Kubectl encapsulates a large number of API calls, making deployment, configuration more simple. Kubernetes-dashboard allows users to operate Kubernetes on the interface without manually entering information such as the call address parameters of each API.

When the API Server receives a deployment request, the Scheduler determines the resource usage of each Node and allocates appropriate nodes to a new container based on the required resources. The criteria include memory, CPU, and disk.

The Controller is responsible for the overall coordination and health of the cluster, ensuring that each component works correctly.

At the bottom of the diagram is the ETCD database. As mentioned above, ETCD is a distributed storage database. As the central database of Kubernetes, ETCD stores the status of the cluster. Components can know the status of the cluster by querying ETCD.

Kubernetes Master allocates containers to nodes for execution. Nodes will be under pressure. Normally, new containers will not run on the Master. Or the Master is unschedulable, but you can also choose to use the Master as a Node, but this is not idiomatic. The following is the architecture diagram of Node:

Kube-proxy manages the network in a Node, and its left and right are critical. Kube-proxy enables pod to POD and pod to Node to communicate with each other by managing iptables. The network is also essentially interworking between pods across hosts.

Kubelet is responsible for reporting information to API Server and storing health status, metrics, and node status information into ETCD.

Docker has been described in detail above and will not be elaborated here.

The Container is designed to ensure that Docker and Kubelet are running while the container is running, while the container is not required to be replaced with a similar container.

A Pod is the smallest deployable cell that can be created and managed in Kubernetes. Multiple containers can be contained in a POD, but Kubernetes only manages pods. If multiple containers are running in a POD, they are running on the same host, so you need to be careful about port occupancy.

References:

K8s.docker8.com/ www.youtube.com/watch?v=zeS…

About the Choerodon toothfish

Choerodon is an open source enterprise services platform that builds on Kubernetes’ container orchestration and management capabilities and integrates DevOps toolchains, microservices and mobile application frameworks to help enterprises achieve agile application delivery and automated operations management. It also provides IoT, payment, data, intelligent insights, enterprise application marketplace and other business components to help enterprises focus on their business and accelerate digital transformation.

Welcome to Choerodon, open source support…

  • Choerodon official website: Choreodon. IO
  • BBS: forum. Choerodon. IO
  • Github:github.com/choerodon/c…
  • Documents: choerodon. IO/useful/docs /

You can also scan the QR code to follow Choerodon on wechat and Weibo: