DUBBO – 2.7 full disclosure

Objective: Learn about the new features in 2.7, and bootleg the version upgrade.

preface

We know that Dubbo opened source in 2011 and stopped updating for a while. Dubbo quietly released version 2.5.4 on GitHub on September 7, 2017. After more than half a year of updates, On February 15, 2018, Dubbo received 14 votes for approval. Without abstention or negative vote, Dubbo became an Apache Foundation incubator project. Now the Dubbo community is very active and the release schedule is very fast.

You can see how active Dubbo is right now.

The Dubbo project now has the following branches:

  1. 2.5.x: This branch was recently voted out of maintenance.
  2. 2.6.x: This branch is still under maintenance and the package name prefix is com.alibaba, which was also contributed to the previous version of Apache.
  3. 2.7.1-release: a temporary branch.
  4. 3. X-dev: It will take Streaming as the core, with key changes in service governance and programming model. I haven’t looked into the details yet, and I will keep track of the changes in this branch, so stay tuned.
  5. Master: the current version is 2.7.x, the package name prefix is org.apache, which is also the development version Dubbo contributed to Apache. The following analysis will also be analyzed in 2.7.1.

Those of you who follow the Dubbo community also know that there was a Meetup in Nanjing on March 23, 2019, in which there was a special topic about the introduction of 2.7 new features. I will explain my own understanding on the basis of the sharer.

The body of the

JDK version

The minimum JDK version required has been changed from 1.6 to 1.8.

(2) Package rename

com.alibaba.dubbo – > org.apache.dubbo

(3) Asynchronous support optimization

We know that the Dubbo protocol itself supports three ways to send requests:

  1. One-way send: Execution methods do not need to return results
  2. Synchronous send: After executing a method, wait for the result to return, otherwise block.
  3. Asynchronous send: When I send a call, I do not block waiting for the result, but return it directly, saving the returned Future into the context for later use. There are two ways to send asynchronously
    1. Future: When the request has a response, future.get() is used to get the response, but future.get() causes the thread to block and the future is retrieved from the RpcContext.
    2. Callback: Sets up a callback thread that executes automatically when a response is received without blocking the current thread.

The 2.6.x version provides asynchronous capabilities, including asynchronous Consumer calls, parameter callbacks, event notifications, and more. However, the current asynchronous approach has the following problems:

  • The Future can only be retrieved in the RpcContext.
  • Future only supports blocking get() interfaces to get results.
  • The ResponseFuture interface does not support automatic callback, and the ResponseFuture interface supports callback callback, but supports limited asynchronous scenarios, such as Future coordination or combination.
  • Provider side asynchrony is not supported

For details, see the dubbo source code analysis (24) in the dubbo protocol to understand the problems.

In 2.7.x, the JDK version was upgraded to 1.8, and the CompletableFuture interface in JDK1.8 was introduced. CompletableFuture supports both Future and callback calls. How CompletableFuture is used in Dubbo will be covered in a future article. After introducing this interface, the following optimizations are made:

  • Supports Provider side asynchrony

  • Support for directly defining the service interface that returns CompletableFuture. With this type of interface, we can more naturally implement asynchronous programming on the Consumer and Provider side.

public interface AsyncService {
    CompletableFuture<String> sayHello(String name);
}
Copy the code
  • If you do not want to define the return value of an interface as a Future type, or if there is a defined synchronous type interface, you can define an additional asynchronous interface and provide methods of type Future.
public interface GreetingsService {
    String sayHi(String name);
}
Copy the code
@AsyncFor(GreetingsService.class)
public interface GrettingServiceAsync extends GreetingsService {
    CompletableFuture<String> sayHiAsync(String name);
}
Copy the code
  • If your original interface definition is not a Future return value, provider-side Async also provides a programming interface similar to Async Servlets in Servlet3.0:RpcContext.startAsync()
public interface AsyncService {
    String sayHello(String name);
}
Copy the code
public class AsyncServiceImpl implements AsyncService {
    public String sayHello(String name) {
        final AsyncContext asyncContext = RpcContext.startAsync();
        new Thread(() -> {
            asyncContext.write("Hello " + name + ", response from provider.");
        }).start();
        return null; }}Copy the code
  • Asynchronous filter chain callback.

The implementation principle will be explained in subsequent articles with the source code. Note that these changes only support the Dubbo protocol.

(4) Transformation of metadata

We know that prior to 2.7, there were only registries, and the registry URL had dozens of key/value pairs that contained all the metadata for a service. As more and more functions are added and metadata becomes extremely large, the following problems arise:

  • The registry stores urls that are too long: This leads to an increase in storage stress, a decrease in notification efficiency after metadata changes due to large data volumes, and increased consumer stress on metadata parsing, especially in large scale scenarios where memory growth is significant
  • The registry takes on too many functions of service governance configuration: synchronization of initial configuration, storage of various run-time configuration rules increases the pressure on the registry, and the flexibility of configuration rules is limited, preventing the integration and expansion of some of the best microservice configuration centers in the market.
  • The function positioning of attributes is not clear: methods, PID and owner are registered attributes for querying services, but these simple information can hardly meet the governance requirements of querying services, so richer registration data is required. For example, methods, although the list of methods is already very long, when OPS developed the service test /mock function, it found that the required data, such as method signatures, were not available.

To solve the above problems, in 2.7, the metadata in the URL is divided into three parts:

  • Metadata information: The complete definition of an interface, including the name of the interface, the methods contained in the interface, and information about the input and output parameters contained in the method. This is important for service testing and service mocks.
  • Execute data on the link: Pass parameters from the provider to the consume so that the consume can perceive them, such as token and timeout
  • Configuration &Ops requirements held by the service itself: Only data required by the provider or consume, such as Executes, Document, etc

After the transformation, three centers were formed respectively:

  • Registries: Ideally, registries will only be used for the synchronization of critical service information (core links), further reducing storage stress on the registry, improving address synchronization efficiency, and alleviating the current Consumer side memory computing stress due to URL redundancy in mass push.
  • Configuration center: To solve the coupling problem of current configuration and address information, through the abstract dynamic configuration layer, developers can connect to more commonly used and professional configuration centers in microservice scenarios, such as Nacos, Apollo, Consul, Etcd, etc. It provides more flexible and diversified configuration rules, including service and application configurations of different granularity, richer routing rules, and dynamic parameter rules for centralized management
  • Service Query Governance Center: For purely service query related data, including Consumer service subscription data, are usually immutable after registration and do not need synchronization between nodes. For example, methods, owner and other keys that can be seen in the current URL and all Consumer urls are currently supported redis (recommended). Zookeeper will serve as the basis for service testing, simulation and other service governance features supported by Dubo-Admin.

(5) Enhanced service governance rules

Routing rule enhancement

Dubbo provides some extensibility routing rules, among which the representative ones are conditional routing and script routing. Problems existing in 2.6.x and the following versions:

  • Routing rules are stored in the registry
  • Only routes of service granularity are supported. Application level cannot define routing rules
  • Route caching is supported, but is basically not scalable
  • Multiple routing rules can be defined for a service or application. Service governance cannot be controlled
  • In implementation, each rule generates a Router instance and loads it dynamically

In version 2.7.x, routing rules have been enhanced:

  • Rich routing rules.
    • Conditional routing: Application level and service level conditions are supported.
    • Marked routing: New introduced to better support traffic isolation, such as grey deployment

Configure the center’s bonus to service governance

  • The separation of governance rules from the registry resulted in the emergence of a configuration center, which made it easier to scale. Apollo and Zookeeper are available, consul and ETCD are also supported in 2.7.1.
  • Application-level dynamic configuration support.
  • Using YAML as the configuration language makes it easier to read and use

(6) Add a configuration center

The configuration center (V2.7.0) has two responsibilities in Dubbo:

  • Externalized configuration: Centralized storage of startup configuration (simply known as external storage of Dubo.properties) One of the purposes of external configuration is to achieve centralized management of configuration. This part of the industry has many mature professional configuration systems such as Apollo, Nacos, etc. The main thing Dubbo does is make sure it works with these systems. The external configuration is no different from other local configurations in terms of content and format, which can be simply understood as the external storage of dubo.properties. The configuration center is more suitable for extracting some common configurations such as registry and metadata center configurations for centralized management

  • Service governance: Storage and notification of service governance rules.

Dubbo supports multiple configuration modes, so it is important to emphasize the priority of configuration coverage, which is reduced from top to bottom:

(7) serialization extension

New Support for Protobuf serialization.

(8) Others

For additional bug fixes and minor tweaks, check out Issues or PR on Github.

Afterword.

For the 2.7.0 boot upgrade, see the following link :dubbo.apache.org/zh-cn/docs/…

This article explains the new features of Dubo2.7. Now that 2.7.1 has been released, those interested can check out what’s new in 2.7.1. In the next article, I will first talk about this asynchronous transformation from the source point of view.