“This is the 18th day of my participation in the First Challenge 2022. For details: First Challenge 2022.”

RPC communication protocol

Dubbo3 provides Triple(Dubbo3), Dubbo2 protocols, which are native to the Dubbo framework. In addition, Dubbo3 also integrates a number of third-party protocols into Dubbo’s programming and service governance architecture, including gRPC, Thrift, JsonRPC, Hessian2, REST, etc. The following focuses on Triple and Dubbo2 protocols.

Triple agreement

Triple protocol is the main protocol introduced by Dubbo3. Triple means the third generation. Through the evolution of Dubbo1.0/Dubbo2.0 protocols and the wave of technology standardization brought by cloud native, Triple, a new Dubbo3 protocol, emerged at the historic moment.

RPC protocol selection

  • Communication protocol is the core of RPC, which regulates the content and format of data transmission in the network.
  • In addition to the required request and response data, it usually contains additional control data, such as the serialization method of a single request, timeout time, compression method, and authentication information.

The content of the agreement includes three parts

  • Data exchange format: Defines the byte stream contents of RPC’s request and response objects as they travel over the network, also known as serialization mode.
  • Protocol structure: Defines the list of fields and the semantics of each field and the arrangement of different fields.
  • Protocols specify how data is transferred across networks by defining rules, formats, and semantics.

A successful RPC requires both ends of communication to be able to read and write network byte streams and convert objects according to protocol. If the two ends cannot agree on the protocol to be used, there will be a chicken fight, which cannot meet the needs of remote communication.

The design of RPC protocol needs to consider the following:

  • Versatility: unified binary format, cross-language, cross-platform, multi-transport layer protocol support
  • Extensibility: Protocols add fields, upgrade, support user extension, and attach business metadata
  • Penetration: can be identified and forwarded by various mid-range devices: gateway, proxy server and other common and high performance usually cannot be achieved at the same time, requires the protocol designer to make certain choices.
  • Performance: As fast As it can be
HTTP / 1.1

Private RPC protocol built directly on the TCP transport layer, remote call solutions built on HTTP will have better versatility, such as: WebService or REST architecture, using HTTP + JSON can be said to be a de facto standard solution.

Choosing to build on top of HTTP has two big advantages:
  • The semantics and extensibility of HTTP meet the requirement of RPC call.
  • HTTP protocol is supported by almost all devices on the network and has good protocol penetration.
Choose to build on TOP of HTTP, but there are some obvious problems:
  • In a typical request-response model, a link can have only one waiting Request at a time. It’s going to produce HOL.
  • Human Readable Headers, which uses a header transmission format that is more general and understandable to humans, but has poor performance
  • There is no direct Server Push support, requiring alternate modes such as Polling long-polling
gRPC

The advantages and disadvantages of constructing RPC protocol on top of HTTP and TCP are as follows: Compared with Dubbo building on top of TCP transport layer, Google chooses to define gRPC directly on top of HTTP/2 protocol. The advantages of gRPC are inherited by HTTP2 and Protobuf.

  • The HTTP2-based protocol is simple enough, low cost for users to learn, and naturally has server push/ multiplexing/flow control capabilities
  • Based on Protobuf multi-language cross-platform binary compatibility ability, providing powerful unified cross-language ability
  • The ecology based on protocol itself is relatively rich, such as k8S/ETCD and other components of the natural support protocol, cloud native de facto protocol standard, but there are some problems
  • The support of service governance is relatively basic, preferring the basic RPC function. The protocol layer lacks the necessary unified definition, and it is not easy for users to use it directly.
  • Protobuf serialization, which is strongly bound, requires a high learning cost and transformation cost, and the migration cost is not negligible for existing users of a partial language

The final choice is Triple

GRPC compatible, with HTTP2 as the transport layer to build a new protocol, namely Triple.

The rise of containerized applications and microservices has led to the development of techniques for content optimization for load. Traditional communication protocols (RESTFUL or other HTTP – based custom protocols) used by clients fail to meet application performance, maintainability, scalability, and security requirements.

The rise of GRPC protocol

A cross-language, modular agreement will gradually become the new application development protocol standard. Since gRPC protocol became a CNCF project in 2017, more and more infrastructure and businesses, including K8S and ETCD, have started to use gRPC ecology. As a cloud native microservization framework, Dubbo’s new protocol is also perfectly compatible with gRPC. In addition, Triple will also enhance and supplement some imperfect parts in gRPC protocol.

Whether Triple protocol solves the problem

  • Performance: Triple protocol adopts the policy of separating metadata and payload. In this way, intermediate devices such as gateways can avoid parsing and deserializing the payload, thus reducing the response time.

  • In terms of routing support, metadata allows users to add custom headers, so that users can easily divide clusters or route routes based on the headers. In this way, when publishing data, cutting stream gray scale or DISASTER recovery (Dr) has higher flexibility.

  • In terms of security, it supports bidirectional TLS authentication (mTLS) and other encrypted transmission capabilities.

  • In terms of ease of use, In addition to supporting Protobuf serialization recommended by native gRPC, Triple supports other serialization such as Hessian/JSON in a general way, which makes it more convenient for users to upgrade to Triple protocol. For the original Dubbo service, modifying or adding Triple protocol only needs to add a line of protocol configuration in the code block declaring the service, and the transformation cost is almost zero.

Triple agreement

Triple protocol communication mode

The status quo

  1. Fully compatible with GRPC, client/server can get through with the native GRPC client

  2. At present, it has been verified by large-scale production practice and reached the production level

Features and Advantages

  1. It has the ability of cross-language communication. Traditional multi-language multi-SDK mode and Mesh cross-language mode both need a more general and easily extensible data transmission format.

  2. Provide a more complete Request model that supports Streaming and Bidirectional in addition to Request/Response.

  3. It is easy to expand and has high penetration, including but not limited to Tracing/Monitoring and other support. It can also be recognized by devices of all layers. Gateway facilities can recognize data packets and are friendly to the deployment of Service Mesh to reduce the difficulty of users to understand.

  4. Multiple serialization methods support smooth upgrade

  5. Support Java user no perception upgrade, do not need to define tedious IDL file, just need to modify the protocol name can easily upgrade to Triple protocol

Triple protocol content

It is further extended based on GRPC protocol

Service-version → Tri-service-version {Dubbo Service Version} service-group → tri-service-group {Dubbo Service Group} Tracing-ID → "tri-trace-traceid" {Tracing ID} Tracing- rpc-id → "tri-trace-rpcid" {_span ID _} cluster-info → "Tri - unit - info" {cluster infomation}Copy the code

GRPC path specifies Service name and method name. GRPC path specifies Service name and method name. Compared with Dubbo protocol, version and group information are missing. Tracing-ID and Tracing- rpc-id represent Tracing ID and SPAN ID respectively. Cluster-info represents Cluster information and can be used to build flexible routing-related service governance capabilities, such as Cluster division.

Triple Streaming

Triple protocol compared with the traditional unARY method, more than the Streaming RPC capability provided by the present

Streaming for what scenarios? In some application scenarios such as large file transmission and live broadcasting, the consumer or provider needs to transmit a large amount of data with the peer end. In these cases, the amount of data is very large, so it is impossible to transmit the data in a SINGLE RPC packet. Therefore, we need to fragment these packets. If we carry out parallel transmission of these split RPC packets through multiple RPC calls, the relevant packets will be disordered after being sent to the peer end, and the received data needs to be sorted and spliced, and the relevant logic will be very complicated. However, if we carry out serial transmission of the split RPC packets, the delay of the corresponding network transmission RTT and data processing will be very large.

In order to solve the above problems and to transmit a large amount of data between consumer and provider in a pipelining way, the Streaming RPC model arises at the right moment.

Through Triple protocol Streaming RPC mode, multiple user-mode long connections (Stream) will be established between consumer and provider. Multiple streams can exist on the same TCP connection at the same time, and each Stream is identified by StreamId. Data packets on a Stream are read and written in sequence.

conclusion

The most important trend in the API space is the rise of standardized technologies. Triple protocol is the main protocol introduced by Dubbo3. It adopts hierarchical design, and its data exchange format is developed based on the Protocol Buffers (Protobuf) Protocol, which has excellent serialization/deserialization efficiency. Of course, it also supports a variety of serialization methods and many development languages. In the transport layer protocol, Triple chooses HTTP/2, and its transmission efficiency has been greatly improved compared with HTTP/1.1. In addition, HTTP/2, as a mature open standard, has rich security, flow control and other capabilities, and has good interoperability. Triple can not only be used for server-side service invocation, but also support the interaction between browsers, mobile apps and IoT devices and back-end services. At the same time, Triple protocol seamlessly supports all service governance capabilities of Dubbo3.

Under the trend of Cloud Native, the requirement of interoperability between cross-platform, cross-manufacturer and cross-environment systems will inevitably give birth to RPC technology based on open standards, and gRPC conforms to the historical trend and has been more and more widely used. In the field of micro services, the proposal and implementation of Triple protocol is a big step for Dubbo3 to move towards cloud native micro services.