This is the sixth day of my participation in the November Gwen Challenge. Check out the event details: The last Gwen Challenge 2021

The control plane is the Sidecar agent that controls and manages the data plane. It implements functions such as configuration distribution, service discovery, traffic routing, and authorization to centrally manage the data plane.

Prior to Istio 1.5, the Mixer component was responsible for ISTIO-Telemetry and ISTIO-Policy, which provided two interfaces, Check and Report, Each time the Sidecar agent sent a request, it checked in the Mixer and reported the observation information to the Mixer, which was processed by the Mixer and sent to Prometheus. As a result, for the sake of performance and ease of deployment, Mixer components were scrapped in Istio 1.5 and these functions were put into the Sidecar.

In Istio 1.5, the control plane is integrated into an independent ISTIOD, which is a single-process, multi-module organization. Istiod is the biggest change in the new version. It replaces the original architecture with a single component, reducing complexity and maintenance while improving ease of use. It is important to note that the original multiple components are not completely removed, but integrated together in the form of modules after reconstruction to form isTIOD.

At present, the control plane still retains the functions of previous components, but is presented in isTIOD in the form of modules, including Pilot, Citadel, and Galley modules, which will be described in detail in this section.

1, the Pilot

Pilot is a core component in Istio that is used to manage and configure all Sidecar agent instances deployed in a specific Istio service grid. It manages routing traffic rules between Sidecar agents and configures failover features such as timeouts, retries, and fuses.

1.1 Pilot architecture

Figure 4.3.1: Pilot architecture diagram

According to the figure above, Pilot has several key modules as follows:

Abstract Model

To support different service registries (e.g., Kubernetes, Consul), Pilot needs a unified storage format for data from different input sources, known as an abstract model.

Key members defined in the abstract model include HostName (Service name), Ports (Service port), Address (Service ClusterIP), and Resolution (load balancing policy).

Platform Adapters

The implementation of Pilot is based on Platform adapters. With Platform adapters, Pilot can convert service registry data to and from an abstract model.

For example, the Kubernetes adapter in Pilot uses the Kubernetes API Server to obtain information about services and pods in Kubernetes, which is then translated into an abstract model for Pilot use.

With the platform adapter pattern, Pilot can also get service information from platforms like Consul, and you can develop adapters to integrate other components that provide service discovery into Pilot.

xDS API

Pilot uses a standard data plane API from the Envoy project to deliver service information and traffic rules to the Sidecar on the data plane. This standard data plane API, also known as xDS.

Sidecar can dynamically obtain Listener, Route, Cluster, Endpoint, and Secret configurations through the xDS API:

  • Typical vmlinux.lds:ListenerDiscover services. The Listener controls the listening of Sidecar startup ports (currently, only TCP is supported) and is configuredL3/L4Layer filter, when the network connection is reached, the configured network filter stack begins to process subsequent events.
  • RDS:RouteThe discovery service is used for the HTTP connection management filter to dynamically obtain the route configuration, including HTTP header modification (adding or deleting HTTP header keys), virtual hosts, and each route entry defined by the Virtual hosts.
  • CDS:ClusterDiscover services used to dynamically obtain Cluster information.
  • EDS:EndpointDiscover services. It is used to dynamically maintain endpoint information, including load balancing weights and canary status, based on which Sidecar can make intelligent load balancing decisions.
  • SDS:SecretDiscovery service, used to dynamically obtain TLS certificates at runtime.

By using this standard API, Istio decouples the control plane from the data plane, opening up the possibility of multiple data plane Sidecar implementations. For example, Ant Financial’s open source Golang version of Sidecar MOSN (Modular Observable Smart Network).

User API (User API)

Pilot also defines a set of user apis that provide high-level business-oriented abstractions that can be understood and used by operations personnel.

O&m personnel use this API to define traffic rules and deliver them to Pilot. These rules are translated into data plane configurations by Pilot and then distributed to Sidecar instances through xDS API. Traffic of microservices can be controlled and adjusted during operation.

By using different flow rules, fine flow control can be carried out for microservices in the grid, such as version shunting, circuit breaker, fault injection, gray publishing, etc.

1.2 Pilot implementation

Figure 4.3.2: Pilot implementation

In the figure, solid lines represent control flow and dotted lines represent data flow. Components with [Pilot] are pilot components. The key components in the figure are as follows:

  • Discovery service: that is,pilot-discoveryTo obtain service information from a registry such as Kubernetes or ConsulKubernetes API ServerTo obtain traffic rules (Kubernetes CRD Resource), and transform the service information and traffic rules into a format that can be processed by the data plane, and deliver it to each Sidecar in the grid through the standard data plane API.
  • Agent: that is,pilot-agentComponent that the process depends onKubernetes API ServerThe configuration information in The Envoy generates an Envoy configuration file that launches and monitors the Sidecar process.
  • Proxy: Sidecar Proxy is a traffic Proxy for all services, directly connectedpilot-discoveryTo indirectly obtain the registration status of microservices in the cluster from service registries such as Kubernetes.
  • Service A/B: For Istio applications, such as Service A/B, the incoming and outgoing network traffic is taken over by the Proxy.

Pilot consists of two components, pilot-Discovery and Pilot-Agent, which are located on the control plane and data plane respectively.

pilot-discovery

Pilot-discovery is located on the control plane and acts as a bridge between service discovery, the control plane and Sidecar. The pilot-Discovery features are as follows:

  • Monitor service registrations in service registries such as Kubernetes. In the Kubernetes environment, resource information such as Service, endpoint, POD, and Node is monitored.
  • Monitor Istio control plane information changes. In Kubernetes, monitor Istio control plane information changesRouteRule,VirtualService,Gateway,EgressRule,ServiceEntrySuch as toKubernetes CRDIs the configuration information about the Istio control plane.
  • Combine the two types of information into configuration information that Sidecar can understand, and provide the information to Sidecar using gRPC protocol.

pilot-agent

The Pilot-Agent is a local agent on the data plane. It is deployed with the Sidecar agent and is responsible for the entire life cycle of the Sidecar service. The pilot-Agent provides the following functions:

  • Generate the Sidecar configuration.
  • Responsible for Sidecar startup and monitoring.

2, the Citadel

Citadel is the core security component responsible for identity authentication and certificate management in Istio. After version 1.5, the independent process is removed and integrated into ISTIOD as a functional module.

2.1 Basic Citadel Functions

In general, Istio’s security architecture includes the following:

  • Certificate Issuing Authority (CA) responsible for key and certificate management.
  • The API server distributes the security configuration to the data plane.
  • Clients and servers communicate securely through proxies.

Istio’s identity model uses a level 1 service identity to identify the source of requests, which can flexibly identify end users, workloads, and so on. At the platform level, Istio can identify itself using something like a service name or directly using the service id provided by the platform. For example, Kubernetes ServiceAccount, AWS IAM users, role accounts, etc.

For identity and certificate management, Istio uses X.509 certificates and supports automatic rotation of keys and certificates. Since version 1.1, Istio has supported secure Discovery Servers (SDS). With continuous improvements and enhancements, VERSION 1.5 SDS has become the default enabled component. Citadel used to do two things: mount certificates in a namespace as Secret; Communicates with the NodeAgent (deprecated) through the SDS gRPC interface. Currently Citadel only needs to do sdS-related work, other functions have been moved to ISTIOD.

2.2 Working Principles of Citadel

Citadel mainly includes CA server, SDS server, certificate key controller and other modules. Their working principles are as follows:

The CA server

The CA signing authority in Citadel is a gRPC server. Two gRPC services are registered at startup. One is the CA service, which is used to process CSR requests (certificate signing request). The other is the certificate service, which issues certificates. The CA first processes the CSR request from the client through the HandleCSR interface, authenticates the client (including TLS authentication and JWT authentication), and then invokes CreateCertificate to issue the certificate.

SDS server

SDS, or Secret Discovery Service, is an API for dynamically obtaining certificate private keys at run time. Envoy proxies dynamically obtain certificate private keys through SDS. The SDS server in Istio is responsible for certificate management and automates security configuration. Compared with traditional methods, using SDS has the following advantages:

  • No need to mount Secret volume;
  • The certificate is dynamically updated without restart.
  • Multiple certificate key pairs can be listened on.

Figure 4.3.3: The Citadel authentication process

In the current version, SDS is enabled by default and its workflow is as follows:

  • Envoy sends certificate and key requests through the SDS API;
  • The IStio-agent, acting as an Envoy, creates a private key and certificate signing request (CSR) and sends it to isTIOD;
  • The CA organization verifies the CSR received and generates the certificate;
  • Istio-agent sends the private key and certificate received from IStiod to Envoy via the SDS API;
  • The above process executes periodically to implement key and certificate rotation.

Certificate key controller

The CaSecretController listens on Secret resources of type IStio. IO /key-and-cert. It periodically checks whether the certificate expires and updates the certificate.

Certificate of rotation

If there is no automatic certificate rotation function, when the certificate expires, you have to restart the issue and restart the agent. Certificate rotation solves this problem and improves service availability. Istio uses a Rotator to automatically check the self-signed root certificate and update it when the certificate is about to expire. Istio is essentially a goroutine polling in the background:

  • Gets the current certificate, resolves the validity period of the certificate and gets the next rotation time.
  • Start the timer. If the certificate reaches the rotation time, obtain the latest certificate key pair from the CA.
  • Update the certificate.

3, Galley

Galley does not directly provide business capabilities to the data plane, but provides support to other modules on the control plane.

As a module responsible for configuration management, Galley verifies the correctness of the format and content of configuration information, and provides these configuration information to the Pilot of the control plane for use. In this way, other modules of the control plane only interact with Galley, thus decoupling from the underlying platform. A simple interface is used to interact with Galley. Galley is responsible for configuration verification, configuration change management, configuration source management, multi-platform adaptation, etc.

\