The company’s internal RPC framework, after a long time of development, has evolved from a completely self-developed to a low-level implementation replaced by Dubbo, but the way of use (API) remains the same. Due to the use of PB serialization protocol, as well as the business code + opcode to define the interface, the development efficiency is very affected, poor understanding, difficult link investigation and other problems, constantly ridiculed by the business side. Hence the third release, which continues to build on the Dubbo extension point and is designed to provide a near-native use of Dubbo.

Since the implementation of the Http RPC protocol provided natively not only uses the API of the Spring framework, but also uses the native serialization of Java, we self-implement the Http RPC protocol based on the extension point, remove the strong dependence on Spring, and use the JSON serialization protocol. The performance test compares the average time and throughput of a single request response between our SELF-implemented Http RPC protocol based on Dubbo extension point and Dubbo’s native Dubbo RPC protocol.

  • Dubbo RPC: Dubbo RPC + Hessian2 serialization protocol
  • Http RPC: Http RPC protocol (jetty for the server, Netty for the client) + JSON serialization protocol (FastJSON)

Purpose of benchmarking

  • At the time of the initial design review, we had some doubts about the performance impact of using HTTP or Dubbo.
  • Compared with the performance data of Dubbo RPC, the performance of self-implemented Http RPC should be optimized to how much is the best, so I have an idea in mind.

Benchmark interface

The interface implementation does not do any business processing, excluding the impact of business time on the benchmark.

RPC interface definition:

public interface DemoeService {
    Result<String> sayHello(String name);
}
Copy the code

Provider implementation:

@ServiceProvider
public class DemoeServiceImpl implements DemoeService {
    @Override
    public Result<String> sayHello(String name) {
        Result<String> result = new Result<>(0, name);
        returnresult; }}Copy the code

Benchmark Test description

1. Register the DemoService service provider with the same configuration and different RPC protocol implementations.

Service provider parameters are as follows:

  • Number of worker threads: 200 (processing business)
  • Number of IO threads: 4 (processing data packet read/write, codec)

2. Create a DemoService service consumer with the same configuration and different RPC protocols.

The only difference is that with the HTTP RPC protocol, connection pooling is required, while with the Dubbo RPC protocol, only a single long connection is configured.

Using the HTTP RPC protocol, the service consumer connection pool configuration:

  • Maximum number of connections: no limit on the maximum number of connections;
  • Maximum number of idle connections: 1024;

3. Use the official open source JMH benchmark testing tool of open-JDK to measure the average time and throughput of DemoService interface respectively.

The environment

  • Locally tested, MacBook Pro 8-core 16GB
  • Close as many redundant processes as possible
  • Start the DuBBo service provider and consumer. After testing the DuBBo protocol, test the HTTP protocol
  • Dubbo version: 2.6.4 (secondary development, added some features)

Benchmark results

Http RPC vs. Dubbo RPC Benchmarks:

  • Measurement indicators: average time and throughput
  • Preheat once, for 5 seconds
  • Take 5 measurements for 5 seconds each
  • Number of consumer threads 200 (simulated concurrency)
  • The packet is small enough (called interface information + parameters, packet less than 1KB)

The first test

Http RPC measurement results:

Benchmark Mode Cnt Score Error Units HttpConsumerBenchmarkTest. TestHttp THRPT 5 to 1.408 + 0.235 ops/ms HttpConsumerBenchmarkTest. TestHttp avgt 5 233.966 + / - 163.614 ms/opCopy the code

Dubbo RPC measurement results:

Benchmark Mode Cnt Score Error Units DubboConsumerBenchmarkTest. TestDubbo THRPT 5 to 42.829 + 24.597 ops/ms DubboConsumerBenchmarkTest. TestDubbo avgt 5 4.739 + / - 0.192 ms/opCopy the code

Comparison of measurement results

throughput The average time
http rpc 1408ops 233.966 ms
dubbo rpc 42829ops 4.739 ms

There is a big difference in the data of this test, and the initial positioning takes time on the server.

After the key steps of printing the diary, it is found that the time of data serialization and deserialization is serious.

The fastJSON serialization and deserialization generics were found to be time-consuming, and the time was reduced by switching to Gson, with performance data close to Dubbo RPC.

Optimize and retest

HTTP RPC (using JSON + gson)

Benchmark Mode Cnt Score Error Units HttpConsumerBenchmarkTest. TestHttp THRPT 5 to 31.550 + 6.477 ops/ms HttpConsumerBenchmarkTest. TestHttp avgt 5 6.487 + / - 1.512 ms/opCopy the code

Dubbo RPC (using hessian2)

Benchmark Mode Cnt Score Error Units DubboConsumerBenchmarkTest. TestDubbo THRPT 5 to 42.829 + 24.597 ops/ms DubboConsumerBenchmarkTest. TestDubbo avgt 5 4.739 + / - 0.192 ms/opCopy the code

Considering that the serialization protocol is also an important factor affecting performance, we added a benchmark using the JSON serialization protocol for Dubbo RPC.

Dubbo (serialization using Fastjson)

Benchmark Mode Cnt Score Error Units DubboConsumerBenchmarkTest. TestDubbo THRPT 5 to 21.416 + 7.571 ops/ms DubboConsumerBenchmarkTest. TestDubbo avgt 5 9.687 + / - 2.329 ms/opCopy the code

Dubbo (serialization using Gson)

Benchmark Mode Cnt Score Error Units DubboConsumerBenchmarkTest. TestDubbo THRPT 5 to 24.460 + 7.396 ops/ms DubboConsumerBenchmarkTest. TestDubbo avgt 5 7.743 + / - 1.256 ms/opCopy the code

As can be seen, the performance difference between Http RPC and Dubbo RPC is between 0.5 ~ 1ms by using json serialization protocol and Gson tool, and the time consumption of Http RPC is slightly lower and the throughput is higher. Dubbo RPC also uses json serialization protocol. The performance difference between Gson and FastJSON is about 2ms, and FastJSON has poor performance.

This test data is for reference only!

Official Performance pressure Test Report: Performance test report