Author: Yu Yu

Yu Yu (Github @Alexstocks), head of the Dubbogo community, is a programmer with ten years of front-line work experience in server-side infrastructure and middleware development. He has successively participated in and improved well-known projects such as Redis/Pika/Muduo/ Dubbo-Go/Sentinel-Go. Currently, I am working on container arrangement in the Trusted Native Department of Ant Financial.

Since the container technology represented by the open source Docker in 2013 and the container choreography technology represented by the open source K8s in 2014 came onto the stage, the relevant technical practitioners accepted it from the perspective of cognition and motion-sense, and the cloud native era has really arrived.

Of course, there are many senior technical personnel believe that the cloud native era should start from the era of virtual machine choreography represented by OpenStack in 2010s. Of course, there are those who say that cloud native technology has been around for a long time, dating back to the days of the mainframe, when minicomputers were created on a supercomputer.

In the era of cloud native, Docker, whose core technology is invariant image, defines the immutable single service deployment form, and K8S, which unifies the container arrangement form, defines the immutable service interface. The combination of the two defines the immutable infrastructure that services can rely on. With this complete and unchanging infrastructure, it is possible to define a new form of immutable middleware — cloud-native middleware.

The middleware in the cloud native era contains immutable cache, communication, message, event and other basic communication facilities. The application can invoke the required service only through the local proxy, without caring about the source of service capability.

1. Microservices framework

From the earliest single application era to the era of distributed technology, the popular technology is microservices. In the era of microservices, major companies have accumulated some representative service communication frameworks, such as Google’s gRPC, Ali’s Dubbo and HSF, Baidu’s bRPC and so on. The competition between multiple service communication frameworks is basically a tug of war between large companies.

From a consumer’s perspective, of course, it is expected that a network framework will continue to evolve with forward compatibility and interoperability between multiple frameworks.

1.1 Backward compatibility of the service framework

The communication framework is based on the communication protocol and serialization protocol, in which a very important issue is the backward compatibility of the new version of the protocol to the old version. Generally, a unified communication framework is used in an organization. However, in reality, due to various reasons, poor backward compatibility of serialization protocol or communication protocol of the same framework may lead to heterogeneous services using different versions of communication framework. If the communication frameworks that use PB V2 and PB V3 are not compatible, neither are the recent (not insignificant) Thrift 0.8.x and Thrift 0.9.x.

Protobuf V3 and Protobuf V2 subversions have good forward and sequential compatibility, but there are some weak internal serialization protocols that are not compatible with each other whether they are TLV or not. Leading to each version of the service framework mutual heterogeneous, eventually lead to use a different version of the service framework of business back a lot of baggage can’t rapid evolution, some kinds of god exist in the new version of the business logic may not compatible with the old version of the business logic, but for communication protocol framework compatible with the old version.

1.2 Interoperability between multiple frameworks

A common sense is that when the size of an organization expands to a large enough degree, it is impossible to have a common framework that can be applied to all scenarios. As the business volume of some large economic entities grows, the business types become large and complex, and there are inevitably some repeated wheels. Any framework suitable for a particular scenario can be developed and maintained affordably and profitably as long as it can find a few landing scenarios internally.

There may be different service frameworks in each branch of the company, which leads to more and more heterogeneous communication among services. For example, there were heterogeneous Dubbo and HSF in Alibaba earlier (currently HSF and Dubbo in Alibaba have begun to merge. HSF has already adopted Dubbo as the kernel and exists in the form of Dubbo plug-in), such as the current network communication between Alibaba’s HSF and various incoming new companies, during which communication may have to resort to the form of Proxy communication gateway.

Backward compatibility between sub-versions of each service framework needs to be maintained, and compatibility between service frameworks may require gateway proxies.

1.3 Interconnection between multilingual frameworks

In addition to the heterogeneity of serialization protocols and communication frameworks, there is also widespread heterogeneity due to the differences between different language technology stacks: each language has its own serialization protocol and communication framework. For example, in the Java world, there are Spring Cloud and Dubbo service governance frameworks. Spring Cloud has not yet been implemented in multiple languages, while Dubbo works better in multiple languages.

1.4 Get through and draw different technical systems

It is fine to have two or three different service frameworks within the same entity between different companies. A large organization can tolerate a moderate amount of duplication, but too much duplication is too much of a good thing. It is said that there are different RPC service frameworks within the giant, and the overall service framework is over 100+!

With the development of domestic Internet industry, due to the obvious head effect tendency, the technology system heterogeneity between large companies and acquired companies is also a problem faced by middleware in the cloud era under the background of big fish eating small fish. Due to the different business scale, the service registry, configuration center, language technology stack, service authentication, cache and storage technologies are not unified between the two. Some of the newly acquired companies even use a rival cloud platform from a large company, which leads to platforming differences, and so on.

With the help of network agent technology, the heterogeneous differences between different technology systems can be preliminarily and quickly solved.

1.5 Necessity of communication agents

In addition to the network access layer agent for north-south communication, in the era of microservices, all service entities using the same communication framework communicate directly with each other. It is rarely heard that all service entities communicate with each other through Proxy, which is based on the homogeneity of the communication framework among all communication entities.

In other words, the network communication agent is digested by various powerful communication frameworks and exists in the form of Proxyless SDK, such as service discovery, service routing, disaster recovery and traffic limiting, load balancing and other functions exist in the SDK of each service framework.

However, with the various dissimilations among various frameworks caused by various factors such as multi-protocol and multi-language, it is generally believed that there is no difference that cannot be solved by a single layer of Proxy.

1.6 Service Mesh

Service Mesh technologies emerging after 2016 can be divided into Proxy Service Mesh and Proxyless Service Mesh. As for the differences between the two, please refer to the humble Opinion on Service Mesh Form in 2019. At present, Proxy Service Mesh is a popular form of Service Mesh technology, and its representative components include envoy on the data surface and Istio on the control surface.

Some Service Mesh technical documentation claims that depositing the serialization protocol, communication capabilities, and Service governance capabilities of the Service framework into proxy of the Service grid (sidecar data surface, such as envoy) has the following benefits:

  • 1 The service framework SDK will become very lightweight, and even completely precipitated into sidecar.
  • The differences between multiple languages, multiple protocols, and multiple communication modes will be smoothed out.
  • 3 Unified flow control to improve system elasticity.
  • 4 unified monitoring facilities to improve the observability.
  • 5 Unified security and trusted authentication improves security.
  • 6 During the upgrade, services are not sensitized, ensuring smooth upgrade and improving reliability.
  • 7 Speed up service version iteration.
  • Quickly get through the differences between different technology governance systems.
  • 9 Quickly switch between Mesh and non-mesh modes.

Some people may mistakenly think that Service Mesh technology can eliminate the complexity of business and Service framework, and the Service grid represented by Istio + Sidecar form can be defined as the ultimate form of Service Mesh.

2 Sidecar and middleware

The sidecar on the data plane of Proxy Service Mesh only has the communication capability. There is still a gap between the Service application and sidecar: the capability of the middleware system used by the application in the micro-service era needs to be deposited into the Sidecar.

2.1 Sidecar capabilities

A typical representative of sidecar in Proxy Service Mesh is envoy, which is a local Proxy with communication capability in essence and has the following data plane capabilities:

  • Flow control
  • Serialization protocol conversion
  • Communication protocol conversion
  • Data routing control

In fact, Envoy merely provides the interface for these capabilities; the specific serialization protocol transformations, communication protocol transformations plug-ins, and so on are left to the infrastructure developers involved.

In addition to serialization protocol and communication protocol transformation, other capabilities of middleware need to do a lot of work on the Envoy Filter layer in the process of sinking to Proxy. Compared with the Proxyless form of Service Mesh, the cost of Proxy Service Mesh has not changed, and its intrusion to business has not been reduced more.

2.2 Protocol Sinking

If the SDK upgrade in the form of Proxyless Service Mesh requires a lot of transformation at the Service level, the transformation cost in the process of upgrading services in the form of Proxy Service Mesh from Proxyless to Proxy is not too large. If Protobuf V3 is used as the communication protocol, the service upgrade protocol in Proxy Service Mesh mode may implement smooth upgrade: Only the Proxy is upgraded, and the service service entity is not upgraded. But the reality is that some proprietary protocols within companies are not backward compatible at all, and the cost is that the Proxy of the service grid is upgraded with the business entity when the protocol is upgraded, and the expected smooth upgrade is difficult to achieve.

The core of these problems is that Local Proxy only focuses on traffic hijacking without unified communication protocol and serialization protocol.

2.3 Stateful Applications

Some advocates of Service Mesh technology assume that the Service entity is a stateless Service and its agent is of course a stateless sidecar, and then advertise the benefits of the Service Mesh era.

As for stateful applications, some people in large companies have the money to spend a huge amount of daily cost to transfer the state to the Proxy to make the application stateless, but smooth upgrade without business awareness is difficult to achieve. This cost is of course affordable for large economic entities and the benefits are very large, but not some small and medium-sized manufacturers can bear, they tread all kinds of pits, may find that they do not have the strength to achieve a state of sidecar.

2.4 Conservation of Complexity

To sum up, building a Proxy Service Mesh based on sidecar may have the following work:

  • Different serialization protocol conversion
  • Conversion between different communication protocols
  • Conversion between multiple versions of the same serialization protocol
  • Conversion between private and public generic serialization protocols
  • The business in the monomer era or micro-service era should be reformed and upgraded
  • Additional operational maintenance resulting from simultaneous upgrades of business applications and sidecar
  • Development, testing and maintenance of stateful Sidecar
  • Elegant exit and smooth upgrade of applications and agents
  • Interoperability between different technology systems
  • Switching between Mesh mode and non-mesh mode

In the face of these difficulties, small and medium-sized manufacturers are likely to retreat to the microservice technology system. The most fundamental reason is that the technical complexity is conserved and the total cost is conserved in the same level of business form. As with microservices technology with individual technologies: individual services are broken down simply enough, but the complexity of individual technologies translates into complexity across huge numbers of services.

3 Another Mesh

The cloud native technology system represented by Service Mesh technology is rooted in the cloud native infrastructure represented by K8S.

3.1 Protocol Standardization

In addition to the passive compatibility of various protocols, cloud native middleware Proxy should take the initiative to unify serialization protocols, such as supporting Protobuf V3 protocol directly, and using this protocol to ensure backward compatibility.

In addition to the passive compatibility of various communication frameworks to do interconnection, can be more active to the de facto communication framework gRPC or communication protocol HTTP close. GRPC has SDK libraries in various languages. Various languages actively provide their own HTTP communication protocol libraries, and HTTP2.0 is backward compatible with HTTP 1.1. For example, Microsoft’s Dapr directly provides the APIS of gRPC and HTTP communication protocols. The upper-layer business needs to choose the API of one of the communication protocols for development.

3.2 the Service Proxy

Protobuf V2 (2008) and Protobuf V3 (2014) unify the serialization protocol. After the release of gRPC V1 in 2016, gRPC gradually became the preferred communication library across languages.

In addition to serialization protocols and communication protocols, the middleware architecture in the era of microservices may include the following technology stacks:

  • RPC, represented by Dubbo/Spring Cloud/gRPC, etc.
  • Flow control such as Hystrix/Sentinel, etc.
  • Cache, which represents Redis.
  • MQ, which stands for Kafka/RocketMQ, etc.
  • Service tracing, such as various frameworks compatible with the Opentracing standard.
  • Collect logs such as Flume/Logtail.
  • Collect indicators, such as Prometheus.
  • Transaction frameworks such as Ali’s Seata.
  • The configuration is delivered, for example, Apollo/NACOS.
  • Service registration, such as ZooKeeper/ETCD.
  • Flow control such as Hystrix/Sentinel etc.
  • Search, such as ElasticSearch.
  • Streaming computing, such as Spark/Flink.

The final state of an immutable middleware facility can be defined retrologically by unifying the various technology stacks into a de facto technical standard. After combing through the above virtual middleware, the overall work is:

  • A standard model for unifying services
  • The multilingual API that defines these standard models
  • A Proxy with standard model apis for communication and middleware
  • Services that fit these apis

The figure above defines a different possible Proxy Service Mesh evolution path than isTIO + Envoy. Sidecar under the Service Mesh model is not only a Local Proxy, but also a Service Proxy that provides standard capabilities of each middleware technology stack.

Service Proxy may be a Proxy that integrates state management, event transmission, message sending and receiving, distributed tracing, search, configuration management, cache data, bypass log transmission and other functions, or it may be a collection of multiple proxies that respectively provide some services. However, the APIS for the individual services provided are unchanged.

3.3 the Application of Mesh

Alternatively, we can split the Service Proxy into two proxies:

  • The existing Local Proxy still exists in the form of a sidecar that hijacks network traffic.
  • Another Application Proxy that provides the individual Service apis.

Application Proxy can be an independent process or container, or it can be an embedded SDK library that can be invoked by an Application. Whether in Proxy or Proxyless form, this is a new form of Service Mesh, which can be called Application Mesh.

Application Proxy can depend on or be independent of Local Proxy. As people often say, level 3 cache is actually an Application Mesh form, from the local cache in the process to the local cache proxy (container, virtual machine or physical machine) and back to the cache cluster. The application only needs to obtain data from the local cache.

Of course, Application Mesh can also be independent of specific infrastructure platforms, including K8S, which will not be covered in this article.

In addition to the benefits of Service Mesh technology, Application Mesh has the following benefits:

  • Better scalability and backward compatibility

    Based on the standard API, the service invariance is greatly improved, which ensures backward compatibility and better scalability of middleware. Based on standard apis, third-party providers can extend more forms of middleware infrastructure on top of the infrastructure provided by cloud vendors.

  • Language independent

    Once the serialization and communication protocols are unified, it is natural to have no specific language dependencies. The mainstream language will support HTTP protocol, gRPC library itself can provide mainstream language support. Another indirect benefit of no language-specific binding is better portability.

  • It has nothing to do with cloud platforms

    By making applications cloud-free through standard services, platforms that unify middleware technology stacks provide the same technical capabilities, and cloud platform users are not in danger of being kidnapped by cloud service providers.

  • More complete decoupling of applications from middleware

    Through standard protocol and standard API, application and middleware are completely decoupled, so that developers can feel the new type of Proxy in the development stage. After the service goes online, the service is not aware of the upgrade process of Proxy and its underlying system, that is, smooth upgrade is better.

  • Application development is simpler

    Based on the standard API, application developers can even use the service framework of the single age to develop distributed applications, improving the speed of application version iteration.

  • Faster startup speed

    Once the state has shifted from the application to the agent, the application can be instantaneously batch started by preheating the agent in advance.

4. Future earnings

No technology is free. Just as microservices technology has brought a dramatic increase in the number of services, Service Mesh technology has brought a reduction in throughput and an increase in latency, but the next cloud-native middleware form will bring a new form of value at an acceptable price.

4.1 Service Value

In terms of its simple technical value, the unification of basic middleware technology can be achieved regardless of language and specific middleware entities. Lighten the burden of business developers, make them focus on business logic, achieve real rapid iteration and smooth upgrade, improve research and development efficiency and reduce various costs.

4.2 Cloud Platform Irrelevant

The business value brought by the new form is that there is no cloud platform dependence. The competition among platforms will not stay on a certain unique core technology advantage, but constantly reduce the service cost under the same technology system to provide better user experience, stronger service capability and more affordable price.

Of course, the organizations that are able and willing to implement such end-state proxy are not small and medium-sized business manufacturers. Therefore, the major cloud vendors that provide these standard services should be under the proxy that has unified these standard service apis. The sooner cloud vendors move to a unified service model, the faster they will benefit, and the more isolated cloud vendors will be those who believe in their own private service capabilities.

4.3 Opportunities for startups

Based on the infrastructure provided by Dafang, an independent service proxy ecosystem can be fostered: some third-party startups are dedicated to providing cloud-native middleware solutions.

Based on the new form of middleware solutions, Low Code or No Code technology can be better implemented. The IDE of the mono-era goes one step further — the IDE of the distributed era, which combines the capabilities of various forms of middleware based on standard apis to develop distributed applications in a WYSIWYG manner.

4.4 Break down the internal barriers of dafang

For some large factory organizations, the real form of Service Mesh technology stack can unify the internal technology stack of large economic entities, break the artificial heterogeneous isolation, improve the efficiency of R&D operation and maintenance and physical resource utilization, and reduce the overall delivery and maintenance cost of human resources and resources.

4.5 Moving towards a New Era

When the cloud native middleware technology system based on the Service Mesh of unified technology is really launched, the Serverless on top of it can have more landing scenarios, the majority of small and medium-sized enterprises can share the technical dividends of the cloud native era, and the coding work of business developers will be less and less. Programming techniques will also get smarter — from manual workshops to automated production by large machines.

If you are interested in the Apache/Dubbo-GO project, you are welcome to join the exchange group.