Service Mesh — Post-Kubernetes microservices

This article is reprinted from: Song Jingchao’s blog

This is not a tutorial, but an attempt to tease out the relationships and interrelationships between Kubernetes, Envoy (xDS protocol), and Istio Service Mesh. This article introduces the load balancing approach in Kubernetes, the Envoy xDS protocol’s significance for Service Mesh, and why Kubernetes requires Istio.

Envoy’s greatest contribution to Service Mesh, or Cloud Native, is the definition of xDS. Envoy is essentially a proxy, but its configuration protocol is supported by many open source software. Such as Istio, Linkerd, AWS App Mesh, SOFAMesh, etc.

About the title of this article

Microservices in a Post-Kubernetes Era Bilgin Ibryam published an article on InfoQ on September 1, 2018. Although the title of this article does not clearly specify what “micro-services in the post-Kubernetes era” is, it can be seen from the article that the author’s point of view is: In the post-Kubernetes era, Service Mesh technology has completely replaced the use of software libraries for network operations (such as Hystrix circuit breakers). This article simply borrows the title.

The following is covered in this article

• This section describes the function of kube-proxy.

• Limitations of Kubernetes in microservice management.

• This section describes the functions of Istio Service Mesh.

• Describe what xDS contains.

• Some concepts in Kubernetes, Envoy, and Istio Service Mesh are compared.

This paper views

If you don’t want to read everything below, here are some of the main points from this article:

• The essence of Kubernetes is application lifecycle management, specifically deployment and management (scaling, automatic recovery, release).

• Kubernetes provides a scalable, highly resilient deployment and management platform for microservices.

• The Service Mesh is based on transparent proxy. The Sidecar proxy intercepts traffic between microservices and manages the behavior of microservices through the configuration of the control plane.

• The Service Mesh decouples traffic management from Kubernetes. Traffic within the Service Mesh does not need the support of the Kube-Proxy component. It manages traffic, security, and visibility between services through an abstraction that is closer to the microservice application layer.

• Envoy xDS defines the protocol standard for Service Mesh configuration.

• The Service Mesh is a higher-level abstraction of the Service in Kubernetes, and the next step is Serverless.

Applicable people

This article assumes that you have a good understanding of Kubernetes and have used Istio Service Mesh, but do not know much about the relationship between Kubernetes, Envoy, and Istio. And how Istio uses the xDS protocol to control all sidecars, so keep reading.

Before reading this article

It is recommended that you have some knowledge of microservices, containers, and Kubernetes before reading this article. It will be helpful if you have already read the following articles, which also refer to some of the ideas in the following articles.

• An in-depth look at the technical details behind Service Mesh by Chao Liu

• An in-depth analysis of Istio traffic management mechanism by Zhao Huabing

• Service Mesh Architecture Rethinking: Where should the boundary between the data plane and the control plane be drawn? By AoXiaoJian

• Understand Envoy proxy Sidecar injection and traffic hijacking in Istio Service Mesh by Song Jinghao

• Service Mesh Deep Learning series — Pilot agent Module analysis for Istio source code analysis by Ding Yiqun

Why did it come to this

Using Service Mesh is not a break with Kubernetes, it’s a natural progression. The essence of Kubernetes is life cycle management of applications through declarative configuration, while the essence of Service Mesh is traffic and security management between applications. If you have built a stable microservice platform using Kubernetes, how do you set up load balancing and traffic control for inter-service calls?

Ask yourself a few questions before reading this article

I think people who have heard of Service Mesh and tried Istio may have the following questions:

1. Why is Istio bound to Kubernetes?

2. What are the roles of Kubernetes and Service Mesh in cloud native?

3. What aspects of Kubernetes does Istio extend? What problems have been solved?

4. What is the relationship between Kubernetes, xDS protocol and Istio?

5. Should I go to the Service Mesh or not?

Why write this article

According to the feedback from the ServiceMesher community, many students who are new to Istio and ServiceMesh can quickly build and try out the various functions of Istio and understand what ServiceMesh can do for you. However, all user operations are performed only on the control plane. How does Istio operate on the data plane, that is, on each Sidecar proxy?

Kubernetes 1.13 was released on schedule, Kubernetes is still iterating at a four-month rate, but we are seeing fewer and fewer major innovations. We have passed the initial stage of technology adoption and are landing on a large scale. I’m not going to talk too much about Kubernetes. This article may not answer all of the above questions, but it is only my personal understanding.

Note: there is a lot of detail about the data plane in this article.

Kubernetes vs Service Mesh

The following figure shows the Service access relationship between Kubernetes and Service Mesh. This paper only applies to sidecar per-Pod mode. For details, please refer to the implementation mode of Service grid.

A Kube-Proxy component is deployed on each node of the Kubernetes cluster. The component communicates with the Kubernetes API Server, obtains the service information in the cluster, and then sets the iptables rules. Requests to a service are directly sent to the corresponding Endpoint (pod belonging to the same set of services).

In Istio Service Mesh, Service in Kubernetes is used for Service registration, and the configuration of data Plane is generated through Control Plane (CRD declaration is used and stored in ETCD). Transparent proxies on the data Plane are deployed in the POD of each application service in the form of a Sidecar container. These proxies request the Control Plane to synchronize the proxy configuration. Because the application container has no aware proxy, the kube-proxy component also needs to intercept traffic, except that kube-proxy intercepts traffic to and from the Kubernetes node, while Sidecar proxy intercepts traffic to and from the Pod. See understanding the routing and forwarding of the Envoy Sidecar proxy in the Istio Service Mesh.

Disadvantages of Service Mesh

Because Kubernetes runs a large number of pods on each node, putting the original Kube-proxy routing and forwarding functionality into each Pod leads to a large number of configuration distribution, synchronization, and final consistency issues. For fine-grained model flow management, a series of new abstractions will be added to the user’s mental burden, but with the popularization of technology will be gradually eased.

Advantages of Service Mesh

The Settings of Kube-proxy take effect globally, so fine-grained control of each Service is not possible. However, the Service Mesh takes the control of traffic in Kubernetes out of the Service layer through sidecar proxy, so more expansion can be made.

Kube – proxy component

In a Kubernetes cluster, each Node runs a Kube-Proxy process. Kube-proxy is responsible for implementing a FORM of VIP (virtual IP) for the Service. In Kubernetes V1.0, the agent is implemented entirely in Userspace. Kubernetes V1.1 added the iptables agent mode, but it is not the default operating mode. As of Kubernetes V1.2, iptables agents are used by default. In Kubernetes V1.8.0-beta.0, ipvS proxy mode was added. For more information about kube-Proxy, see Introduction to Kubernetes: Principles of Service and Kube-Proxy and Implementation of Kubernetes Import Traffic Load Balancing using IPVS.

The defects of kube – proxy

In the link above the author points out the shortcomings of Kube-proxy:

First, there are LIVENESS probes that can be addressed if a forwarded POD is not serving properly, it will not automatically try another POD. Each POD has a health check mechanism, when a POD health problem, Kube-Proxy will delete the corresponding forwarding rule. In addition, nodeport-type services cannot add TLS or more complex packet routing mechanisms.

Kube-proxy implements load balancing among pod instances of Kubernetes Service, but how to do fine-grained control over the traffic among these services, such as dividing the traffic by percentage to different application versions (these applications belong to the same service, But located on a different Deployment), do canary releases (grayscale releases) and blue and green releases? The Kubernetes community provides a way to use Deployment for canary publishing, which is essentially to assign different pods to A Deployment Service by changing the pod label.

Kubernetes Ingress vs Istio Gateway

The Ingress resource object in Kubernetes has the same function as the Gateway in Istio Service Mesh. The Ingress resource object is responsible for the north-south traffic (incoming traffic from outside the cluster) in the cluster.

Kube-proxy can only route the traffic inside Kubernetes cluster, and we know that the Pod of Kubernetes cluster is located in the external network created by CNI, and the external cluster cannot directly communicate with it. Therefore, Kubernetes creates the ingress resource object, which is driven by the Ingress Controller at the edge of the Kubernetes node (which can be multiple or a group of nodes). Ingress is responsible for managing north-south traffic (traffic entering the Kubernetes cluster from outside the cluster). Ingress must be connected to various Ingress controllers, such as Nginx Ingress Controller and Traefik. Ingress only works with HTTP traffic and is simple enough to route traffic by matching only limited fields such as Service, port, and HTTP path. This makes it impossible to route TCP traffic such as MySQL, Redis, and various private RPCS. To directly route north-south traffic, you can only use the LoadBalancer or NodePort of a Service, which requires cloud vendor support and may be paid for, or the NodePort of a Service, which requires additional port management. Some Ingress controllers support exposure of TCP and UDP services, but can only be exposed using services. The Ingress itself is not supported, for example, the Nginx Ingress Controller, The exposed ports of the service are configured by creating a ConfigMap.

The load balancer described by Istio Gateway is used to host connections to and from the edge of the grid. This specification describes a series of open ports, the protocols used by these ports, SNI configuration for load balancing, and so on. Gateway is a CRD extension that also reuses the capabilities of Envoy Proxy, as detailed on the Istio website.

XDS agreement

In the following diagram, you may have seen the Service Mesh before. Each square represents an example of a Service, such as a Pod in Kubernetes that contains a Sidecar proxy. The xDS protocol controls the specific behavior of all traffic in the Istio Service Mesh, as shown in the following figure.

The xDS protocol was developed by Envoy, and the original xDS protocol in the Envoy V2 version API only refers to CDS, EDS, LDS, and RDS.

Let’s take an example of two Services, each with two instances of the xDS protocol Envoy.

The arrows in the figure above are not the path or route of the traffic after it enters the Enovy Proxy, but rather an imaginary order in which xDS interfaces deal with each other rather than the actual order, which is actually cross-referenced between xDS.

Envoy dynamically finds resources by querying files or managing servers. Broadly speaking, the corresponding discovery service and its corresponding API are called

Subscribe (

subscription

)

• File subscription: Monitors files under a specified path. The easiest way to discover dynamic resources is to save them to a file and configure the path in the ConfigSource parameter.

• GRPC streaming Subscription: Each xDS API can be individually configured with an ApiConfigSource pointing to the cluster address of the corresponding upstream management server.

• Polling REST-JSON polling subscription: Synchronous (long) polling of REST endpoints that can be done by a single xDS API.

For details about the xDS subscription mode, see xDS Protocol Resolution. Istio uses gRPC streaming subscription to configure the Sidecar proxy for all data planes.

For a detailed breakdown of xDS protocol, please refer to these articles by Dr. Ding Yiqun:

• Service Mesh deep learning series part1 – istio source code analysis of the pilot-agent module analysis

• Service Mesh deep learning series part2 — pilot-discovery module analysis of istio source code

• Service Mesh Deep Learning Series part3 — IsTIO Source Code Analysis by Pilot-Discovery Module (Continued)

This article introduces the overall architecture of Istio Pilot, Envoy configuration generation, the functions of the Pilot-Discovery module, as well as CDS, EDS and ADS in xDS protocol. Please refer to the official Enovy documentation for details on ADS.

XDS protocol essentials

Finally, summarize the key points of xDS protocol:

• CDS, EDS, LDS and RDS are the most basic xDS protocols, which can be updated independently.

• All Discovery services can connect to different Management Servers, which means that multiple servers can manage xDS.

• The Envoy has made a series of extensions to the original xDS protocol, adding apis such as SDS (Secret Key Discovery Service), ADS (Aggregated Discovery Service), HDS (Health Discovery Service), MS (Metric Service), RLS (Rate limiting service), etc.

• To ensure data consistency, if you use the xDS raw API directly, you need to ensure that the order of updates is as follows: CDS –> EDS –> LDS –> RDS, which follows the make-before-break principle of electrical engineering, that is, establish a new connection Before disconnecting the original one. When a new routing rule is set, traffic is discarded because the upstream cluster cannot be discovered, which is similar to circuit disconnection.

• CDS sets which services are in the Service Mesh.

• EDS sets which instances (endpoints) belong to these services (clusters).

• LDS sets the port to listen on the instance to configure the route.

• RDS The routing relationship between the final services should ensure that the RDS is updated last.

Envoy

Envoy is the default Sidecar in the Istio Service Mesh. Istio extends its control plane based on Enovy in terms of Envoy xDS protocol. Before we get to the Envoy xDS protocol we need to familiarize ourselves with the basic terms Envoy. Here’s a list of the basic terms and data structures in Envoy. For details on Envoy, please refer to the official Envoy documentation, For details on how envoys work as forwarding proxies in Service Mesh (not just Istio) please refer to this article by Liu Chao on netease Cloud for an in-depth look at the technical details behind Service Mesh and how to understand Service Proxies in Istio Service Mesh Sidecar injection and traffic hijacking, some of which are cited in this article.

Basic terminology

Here are the basic terms you should know about Enovy:

• Downstream: A Downstream host connects to an Envoy, sends a request and receives a response, that is, the host that sent the request.

• Upstream: An Upstream host receives a connection and request from an Envoy and returns a response, i.e. the host receiving the request.

• Listeners: Listeners are named network addresses (for example, ports, Unix Domain sockets, etc.) that downstream clients can connect to. Envoy exposes one or more listeners to the downstream host for connection.

• Cluster: A Cluster is a set of logically identical upstream hosts connected by an Envoy. Envoy finds cluster members through service discovery. You can choose to perform an active health check to determine the health status of cluster members. Envoy determines which member of the cluster to route the request to using a load-balancing policy.

Envoys can set multiple listeners, each with a filter chain, and the filters are extensible, making it easier to manipulate the behavior of traffic, such as setting encryption, private RPCS, etc.

The xDS protocol was presented by Envoy and is now the default Sidecar proxy in Istio, but as long as you implement xDS you can theoretically use it as a Sidecar proxy in Istio, For example, Ant Financial open-source SOFAMosn and Nginx open-source NginMesh.

Istio Service Mesh

Istio is a rich Service Mesh, which includes the following functions:

• Traffic management: This is the most basic function of Istio.

• Policy control: Through Mixer components and various adapters, access control systems, telemetry capture, quota management, and billing are implemented.

• Observability: Through Mixer.

• Security authentication: The Citadel component manages keys and certificates.

Traffic management in Istio

Istio defines the following CRDS to help users manage traffic:

• Gateway: Gateway describes a load balancer that operates on the edge of a network to receive incoming or outgoing HTTP/TCP connections.

• VirtualService: VirtualService actually connects the Kubernetes service to Istio Gateway. It can do more, such as defining a set of traffic routing rules that can be applied when a host is addressed.

• DestinationRule: Indicates the policy defined by the DestinationRule, which determines the access policy for the traffic after routing. Simply put, it defines how traffic is routed. These policies can define load balancing configurations, connection pool sizes, and external detection configurations for identifying and evicting unhealthy hosts from the load balancing pool.

• EnvoyFilter: The EnvoyFilter object describes the filters for the broker service that can be customized to the broker configuration generated by Istio Pilot. This configuration is rarely used by novice users.

• ServiceEntry: By default, services in an Istio Service Mesh cannot discover services outside the Mesh. ServiceEntry adds additional entries to the internal Service registry in Istio so that automatically discovered services in the Mesh can access and route to manually added services.

Kubernetes vs Envoy xDS vs Istio

After reading the abstract concepts of traffic management in Kubernetes’ Kube-Proxy component, Envoy xDS, and Istio above, the following will take you through a comparison of the three corresponding components/protocols in terms of traffic management only (note that the three cannot be identical).

conclusion

If the object managed by Kubernetes is Pod, then the object managed by Service Mesh is one Service. Therefore, it is natural to use Kubernetes to manage micro services and then apply Service Mesh. If you don’t care about services, use a Serverless platform like Knative. That’s another story.

This concept is just the tip of the iceberg in Istio’s new layer of abstraction on top of Kubernetes. However, traffic management is one of the most fundamental and important functions of a service grid, so this article starts here and will give you more details about the service grid in the future. Please follow my blog jimmysong. IO and istio-handbook.

reference

• Basic concepts of Istio traffic management – jimmysong. IO

• Kubernetes kube-proxy iptables proxy mode – jimmysong. IO

• Kubernetes kube-proxy ipvS proxy mode – jimmysong. IO

• Envoy V2 API Overview – Servicemesher.com

• Listener Discovery Service (LDS) – servicemesher.com

• Route Discovery Service (RDS) – servicemesher.com

• Cluster Discovery Service (CDS) – servicemesher.com

• Kubernetes service – jimmysong.io

• XDS protocol parsing – jimmysong. IO

• Dive into the technical details behind Service Mesh – CNblogs.com

• Understand Envoy proxy Sidecar injection and traffic hijacking in Istio Service Mesh – jimmysong. IO

• Kubernetes service and Kube-Proxy – cizixs.com

• Implementation of Kubernetes entry traffic load balancing using IPVS – jishu. IO

• Zhaohuabing.com – In-depth analysis of Istio Traffic management mechanism

• The road to enterprise service Grid Architecture – Jimmysong. IO

• Debug Envoy and pilot-istio.io