For developers, asynchrony is an idea of programming. Programs designed with asynchrony can significantly reduce thread wait, thus greatly improving the overall performance of the system and significantly reducing latency in high-throughput scenarios.

Therefore, middleware systems such as message queue, which need ultra-high throughput and ultra-low latency, will adopt asynchronous design ideas in their core processes.

Let’s take a look at a very simple example of how using asynchronous design can improve system performance.

How does asynchronous design improve system performance?

Suppose we want to implement a microservice Transfer(accountFrom, accountTo, amount) for a money Transfer. This service takes three parameters: the accountTo be transferred out, the accountTo be transferred, and the amount to be transferred.

The implementation process is also relatively simple, we need to transfer 100 yuan from account A to account B:

1. Subtract $100 from A’s account;
2. Add 100 yuan to B’s account and the transfer is complete.

The corresponding sequence diagram looks like this:

During the implementation of this example, we called another microservice Add(Account, amount), which adds amount to the account and, when amount is negative, deducts the amount of the response.

In particular, in this code I have omitted error handling and transaction-related code in order to keep things simple so that we can focus on asynchrony and performance optimization. You should not do this in real development.

Performance bottlenecks for synchronous implementations

First, let’s take a look at the synchronization implementation, corresponding pseudocode is as follows:

 

Transfer(accountFrom, accountTo, amount) {Add(accountFrom, accountTo, amount) {Transfer(accountFrom, accountTo, amount) { -1 * amount) // Add(accountTo, amount) return OK}Copy the code

The pseudocode above subtracts the corresponding amount of money from the accountFrom account and adds the subtracted amount to accountTo’s account in a natural, straightforward way. So what about performance? Let’s analyze the performance together.

Assuming that the average response delay of microservice Add is 50ms, it is easy to calculate that the average response delay of microservice Transfer we achieved is approximately equal to the delay of executing Add for 2 times, namely 100ms. What happens as more and more requests are made to invoke the Transfer service?

In this implementation, where each request takes 100ms to process and a thread is exclusive for 100ms, one can conclude that each thread can process up to 10 requests per second. We know that thread resources on each computer are not infinite. Assuming we are using a server with a maximum number of open threads of 10,000, we can calculate the maximum number of requests that the server can handle per second: 10,000 (threads) *10 (requests per second) = 100,000 (requests per second).

If the request speed exceeds this value, the request cannot be processed immediately and can only be blocked or queued. At this time, the response delay of Transfer service is extended from 100ms to: queuing waiting delay + processing delay (100ms). In other words, the average response time of our microservice became longer in the case of a large number of requests.

Is this the limit of what this server can handle? In fact, far from it. If we monitor various indicators of the server, we will find that no matter CPU, memory, network card traffic or disk IO are idle. Then what are the 10,000 threads in our Transfer service doing? Yes, most threads are waiting for the Add service to return results.

That is, with a synchronous implementation, all threads throughout the server are not working most of the time, but are waiting.

If we can reduce or avoid this kind of meaningless waiting, we can greatly improve the throughput of the service and thus the overall performance of the service.

An asynchronous implementation is used to solve the wait problem

Let’s look at how we can solve this problem with asynchronous thinking and implement the same business logic.

 

TransferAsync(accountFrom, accountTo, Amount, OnComplete()) {// Asynchronously subtract the corresponding amount of money from the accountFrom account and call OnDebit. AddAsync(accountFrom, -1 * amount, OnDebit(accountTo, amount, OnAllDone(OnComplete())} // Call OnDebit(accountTo, amount, OnAllDone(OnComplete())) {asynchronously add the subtracted amount to accountTo's account, Then execute the OnAllDone method AddAsync(accountTo, amount, OnAllDone(OnComplete()))} // Call OnAllDone(OnComplete()) {OnComplete()}Copy the code

You may have noticed that the TransferAsync service takes one more parameter than Transfer, and that this parameter is passed as a callback method OnComplete() (although the Java language does not support passing methods as method parameters, But many languages, such as JavaScript, have this feature, and in the Java language, you can do something similar by passing in an instance of a callback class).

The semantics of this TransferAsync() method are: Please do the transfer for me, and when the transfer is complete, call OnComplete(). The thread calling TransferAsync can return immediately without waiting for the transfer to complete. When the transfer is complete, the TransferService will call the OnComplete() method to complete the transfer.

Asynchronous implementation is a bit more complicated than synchronous. Let’s first define two callback methods:

• OnDebit() : callback method called after the accountFrom deduction is complete;
• OnAllDone() : callback method called after accountTo is completed.

The semantics of the entire asynchronous implementation are equivalent to:

1. Asynchronously subtract the corresponding amount of money from the accountFrom account and call the OnDebit method;
2. In the OnDebit method, asynchronously add the subtracted amount to accountTo’s account, and then execute the OnAllDone method;
3. In the OnAllDone method, the OnComplete method is called.

The sequence diagram looks like this:

As you can see, the timing of the entire process is exactly the same in the asynchronous implementation as in the synchronous implementation, except for the mechanism for asynchronous calls and callbacks instead of synchronous sequential calls in the threaded model.

Next, let’s analyze the performance of the asynchronous implementation. Since the timing of the process is the same as that of the synchronous implementation, the average response latency is 100ms in the case of low number of requests. In the case of high number of requests, the asynchronous implementation no longer needs threads to wait for execution results, but only needs a number of threads to achieve the same throughput as a large number of threads in the synchronous scenario.

Since there is no limit on the number of threads, the overall throughput upper limit will greatly exceed that of synchronous implementation, and before server CPU and network bandwidth resources reach the limit, the response latency will not increase significantly with the increase of the number of requests, and can almost keep the average response latency of about 100ms.

See, that’s the magic of asynchrony.

Simple and practical asynchronous framework: CompletableFuture

In practical development, we can use asynchronous and reactive frameworks to solve some common asynchronous programming problems and simplify development. The most commonly used asynchronous frameworks in Java are CompletableFuture built in Java8 and RxJava of ReactiveX. I personally prefer simple, practical and easy to understand CompletableFuture, but RxJava has more powerful functions. Those of you who are interested can take a closer look.

A very powerful new class for asynchronous programming has been added to Java 8: CompletableFuture, which captures most of the functionality we use to develop asynchronous programs, making it easy to write elegant and maintainable asynchronous code with CompletableFuture.

Next, let’s look at how to implement a money transfer service with CompletableFuture.

First, we use CompletableFuture to define the interfaces for two microservices:

 

/** * Account service */ public interface AccountService {/** * Change account amount * @param Account ID * @param amount Increase amount, */ CompletableFuture<Void> add(int account, int amount); }Copy the code

 

/ * * * money transfer service * / public interface TransferService {/ * * * * @ asynchronous transfer service param fromAccount transfer account * @ param * @ param toAccount into account */ CompletableFuture<Void> Transfer (int fromAccount, int toAccount, int amount); }Copy the code

You can see that the return type of the method defined in both interfaces is a CompletableFeture with a generic type. The generic type in Angle brackets is the type that the real method needs to return data. Our two services don’t need to return data, so we can just use Void.

Then we implement the transfer service:

 

*/ Public class TransferServiceImpl implements TransferService {@inject private AccountService accountService; @override public CompletableFuture<Void> Transfer (int fromAccount, int toAccount, Int amount) // Call add asynchronously to deduct the corresponding amount from fromAccount ThenCompose (v -> accountService.add(toAccount, amount)); thenCompose(v -> accountService.add(toAccount, amount)); }}Copy the code

In the transfer service implementation class TransferServiceImpl, an AccountService instance is defined. This instance is injected from the outside. How to inject is not our concern, but we will assume that the instance is available.

Then we look at implementing the Transfer () method. We call the accountService.add() method once to deduct the response from fromAccount because add() returns a CompletableFeture object, The thenCompose() method of CompletableFeture can be used to concatenate the next call to accountservice.add () to make two asynchronous calls to the accountService in sequence to complete the transfer.

The client is also very flexible with CompletableFuture, which can be called either synchronously or asynchronously.

 

public class Client { @Inject private TransferService transferService; Private final static int A = 1000; private final static int A = 1000; private final static int B = 1001; Public void syncInvoke() throws ExecutionException, InterruptedException {// Call transferService.transfer(A, B, 100).get(); System.out.println(" Transfer completed!" ); } public void asyncInvoke() {transferService.transfer(A, B, 100).thenRun(() -> system.out.println (" Transfer completed! )); }}Copy the code

After calling an asynchronous method to get the return value of the CompletableFuture object, you can either call the Get method of the CompletableFuture and wait for the completion of the called method to execute and get the return value as you would call a synchronous method, or you can call the CompletableFuture method in the same way as an asynchronous callback. Call the CompletableFuture series of methods that begin with THEN, and define subsequent actions for the CompletableFuture after the asynchronous method ends. For example, in the example above, we call thenRun() and the argument is to print the transfer to the console. This allows us to print “Transfer completed!” on the console after the transfer is completed. .

conclusion

In simple terms, the idea of asynchrony is that when we want to perform a time-consuming operation, instead of waiting for the operation to finish, we give the operation a command: “When the operation is finished, what should we do next?”

Using the asynchronous programming model, although it does not speed up the program itself, it can reduce or avoid thread waits, and only a few threads can achieve high throughput.

There are also problems with the asynchronous model: asynchronous implementations are much more complex than synchronous implementations, and code readability and maintainability are significantly reduced. Although the use of some asynchronous programming frameworks can simplify asynchronous development to some extent, it does not solve the problem of high complexity of asynchronous models.

Asynchronous performance is good, but it should not be abused. Use the asynchronous model only in scenarios where the business logic is simple and requires very high throughput, such as message queuing, or where you have to wait for resources for a long time. If the business logic of the system is complex, it is wise to adopt a synchronous model that conforms to human nature and is easy to develop and maintain when the performance is sufficient to meet the business requirements.

Author: Mr. Murong kiloword links: www.jianshu.com/p/906a9a366… The copyright of the book belongs to the author. Commercial reprint please contact the author for authorization, non-commercial reprint please indicate the source.