Talk about how to make your project asynchronous.
1. Synchronous and asynchronous, blocking and non-blocking
Synchronous and asynchronous, blocking and non-blocking, this word has been a platize, at that time there are often many students do not know clearly, think that synchronous must be blocking, asynchronous must be non-blocking, other they are not the same thing.
Synchronous and asynchronous are concerned with the communication mechanism of the resulting messages
Synchronous: Synchronous means that the caller does not actively wait for the result to return. Asynchronous: Asynchronous means that the caller does not actively wait for the result to return, but by other means such as status notification, callback function, etc. Blocking and non-blocking are primarily concerned with waiting for the result to return to the caller’s state
Blocked: the current thread is suspended and does nothing until the result is returned. Non-blocked: the thread can do something else without being suspended until the result is returned. You can see synchronous and asynchronous, blocking and non-blocking, the main points of concern are different, and some people ask can synchronous be non-blocking, can asynchronous be blocking? Of course it can. Here are a few simple examples to better illustrate the meanings between their combinations: 1. Synchronous blocking: Synchronous blocking is basically the most common model in programming. For example, you go to the store to buy clothes. When you get there, you find that the clothes are sold out, and you wait in the store, doing nothing (including looking at your phone), until they are available.
2. Synchronous non-blocking: Synchronous non-blocking can be abstracted as a polling mode in programming. When you go to the store and find that the clothes are sold out, you don’t need to wait stupidly.
3. Asynchronous blocking: Asynchronous blocking is used less frequently in this programming. It is similar to writing a thread pool, submitting and future.get() immediately, so that the thread is still suspended. It’s kind of like when you go to the store to buy clothes, and when you find the clothes are gone, you give the boss a phone number and say to call me when the clothes arrive, and then you wait for the phone to ring and do nothing. It does feel silly, so this mode is used less often.
4. Asynchronous non-blocking: Asynchronous non-blocking this is also a core of highly concurrent programming today, and it’s a core of today’s focus. It’s like when you go to the store to buy clothes and they’re gone, all you have to do is tell the boss this is my phone number and call when the clothes arrive. Then you can go and play as you like, and you don’t have to worry about when the clothes will arrive, and when the phone rings, you can go and buy them.
2. Synchronous blocking versus asynchronous non-blocking
We have already seen how inefficient synchronous blocking is. If you use synchronous blocking to buy clothes, you may only buy one piece of clothes in a day and do nothing else. If you use asynchronous non-blocking to buy clothes, it is just a small thing that you do in a day.
Our code, the map to us when our thread a RPC calls or HTTP invocation, or some other time consuming IO call, after launch, if it is a synchronized block, we this thread will be blocked hangs, until the results back, imagine if IO call very frequent that our CPU utilization rate is very low is very low. So-called is everything, now that the CPU utilization rate is low, having a lot of IO call that we can use asynchronous non-blocking, when called in IO I don’t care about the results immediately, I just need to write a callback function to the IO call, this time I thread can continue to process new requests, when IO at the end of the end of the call, will call the callback function. Our thread is always busy, so we can do more meaningful things.
First of all, asynchrony is not a panacea. Asynchrony does not shorten the call time of your entire link, but it can greatly improve your maximum QPS. In general, there are two parts of our business that are time-consuming:
CPU: CPU time refers to our general business processing logic, such as some data operations, object serialization. This asynchronization cannot be solved, but requires some algorithm optimization or some high-performance framework. As mentioned above, iowait occurs during network calls, file transfers, etc., when threads are suspended and blocked. Our asynchrony is usually used to solve this part of the problem. 3. What can be asynchronized?
As mentioned above, asynchrony is used to solve the problem of IO blocking, and we can use asynchrony in general projects as follows:
Servlet asyncization, SpringMVC asyncization RPC calls like dubbo (THRIFT), HTTP calls asyncization database calls, cache calls asyncization and I’m going to talk about asyncization in a couple of ways.
4. The servlet asynchronous
Servlets are no stranger to Java developers, and whether you use Struts2 or SpringMVC in your projects, they are essentially packaged servlets. But our general development, in fact, is the use of synchronous blocking mode as follows:
The advantages of the above mode are that it is easy to code and suitable for projects with less traffic or a lot of CPU operations in the initial stage of the project
The disadvantage is that the business logic thread and the servlet container thread are the same, the general business logic has to occur IO, such as database query, such as RPC call, this time will block, and our servlet container thread must be limited. When servlet container threads are blocked, our service will be denied access. Otherwise, we can add a number of machines to solve this problem, but as the saying goes, it is better to rely on others to share the request than to handle it myself. So after servlet3.0 support for asynchrony, we use asynchrony to look like this:
Here we use a new thread to handle the business logic, so that the BLOCKING of IO calls does not affect our SERlVET, and the code to implement asynchronous SERlVET is relatively simple, as follows:
@WebServlet ( name
“WorkServlet” , urlPatterns
“/work” , asyncSupported
true ) public
class
WorkServlet
extends
HttpServlet {
private
static
final
long serialVersionUID
1L ;
@Override
protected
void doGet ( HttpServletRequest req ,
HttpServletResponse resp )
throws
ServletException ,
IOException
{
this . doPost ( req , resp );
}
@Override
protected
void doPost ( HttpServletRequest req ,
HttpServletResponse resp )
throws
ServletException ,
IOException
{
SetContentType (“text/plain; charset=UTF-8” ); resp . setHeader ( “Cache-Control” , “private” ); resp . setHeader ( “Pragma” , “no-cache” );
final
PrintWriter writer
resp . getWriter (); Writer. println (” Teacher checked homework “); writer . flush ();
List < String
zuoyes
new
ArrayList < String
(a);
for
( int i
0; i <
10; i ++)
{ zuoyes . add ( “zuoye” + i );;
}
// Enable asynchronous requests
final
AsyncContext ac
req . startAsync (); doZuoye ( ac , zuoyes ); Writer. println (” teacher assign homework “); writer . flush ();
}
private
void doZuoye ( final
AsyncContext ac ,
final
List < String
zuoyes )
{ ac . setTimeout ( 1 * 60 * 60 * 1000L ); ac . start ( new
Runnable ()
{
@Override
public
void run ()
{
// Get the character output stream through response
try
{
PrintWriter writer
ac . getResponse (). getWriter ();
for
( String zuoye : zuoyes )
{writer. println (“”” + zuoye + “”” “);
Thread . sleep ( 1 * 1000L ); writer . flush ();
} ac . complete ();
}
catch
( Exception e )
{ e . printStackTrace ();
}
}
});
}} The key to serlvet is that HTTP takes a long connection, meaning that when a request comes in, it does not close, even if it returns, because it may still have data until a close command is returned. AsyncContext ac=req.startAsync(); It is used to get the asynchronous context, and then we call back the data through the asynchronous context, kind of like when we buy clothes, we call the boss, and this context is also a phone, and when the clothes arrive, that is, when the data is ready, we can call and send the data. ac.complete(); Used to close long links.
In fact, very few people now do serlvet programming, are directly using some ready-made frameworks, such as Struts2, SpringMVC. Here’s how to do asynchrony with SpringMVC:
First make sure the servlets in your project are 3.0 or above!! , the second springMVC4.0 +
“3.0”
xmlns
“Java.sun.com/xml/ns/java…”
xmlns:xsi
“Www.w3.org/2001/XMLSch…”
xsi:schemaLocation
“Java.sun.com/xml/ns/java… Java.sun.com/xml/ns/java…”
Using SpringMVC encapsulates AsyncContext for servlets, which is relatively simple to use. Previously, the Controller of our synchronized mode was return ModelAndView, RequestMapping (value? RequestMapping (value
“/asynctask” , method
RequestMethod . GET )
public
DeferredResult < String
asyncTask ()
throws
IOReactorException
{
IOReactorConfig ioReactorConfig
IOReactorConfig . custom (). setIoThreadCount ( 1 ). build ();
ConnectingIOReactor ioReactor
new
DefaultConnectingIOReactor ( ioReactorConfig );
PoolingNHttpClientConnectionManager conManager
new
PoolingNHttpClientConnectionManager ( ioReactor ); conManager . setMaxTotal ( 100 ); conManager . setDefaultMaxPerRoute ( 100 );
CloseableHttpAsyncClient httpclient
HttpAsyncClients . custom (). setConnectionManager ( conManager ). build ();
// Start the client httpclient . start ();
// Set the timeout period to 200ms
final
DeferredResult < String
deferredResult
new
DeferredResult < String
( 200L ); deferredResult . onTimeout ( new
Runnable ()
{
@Override
public
void run ()
{
System.out. println (” Asynchronous call execution timed out! thread id is : “
Thread . currentThread (). getId ()); DeferredResult. SetResult (” timed out “);
}
});
System . out . println ( “/asynctask 调用！thread id is : “
Thread . currentThread (). getId ());
final
HttpGet request2
new
HttpGet ( “www.apache.org/” ); httpclient . execute ( request2 ,
new
FutureCallback < HttpResponse
(a)
{
public
void completed ( final
HttpResponse response2 )
{
System . out . println ( request2 . getRequestLine ()
“- >”
response2 . getStatusLine ()); deferredResult . setResult ( request2 . getRequestLine ()
“- >”
response2 . getStatusLine ());
}
public
void failed ( final
Exception ex )
{
System . out . println ( request2 . getRequestLine ()
“- >”
ex );
}
public
void cancelled ()
{
System . out . println ( request2 . getRequestLine ()
” cancelled” );
}
});
return deferredResult ;
} Note: a problem with serlVET asynchronization is that filter postprocessing cannot be used. For some of our methods, serlVET asynchronization cannot be used directly. This problem is solved in SpringMVC, which takes a more subtle approach to request forwarding, allowing the request to be filtered again. However, a new problem is introduced that the filter will be processed twice. Here we can use SpringMVC source code to determine its own method, we can use the following statement in the filter to determine whether the request is forwarded by SpringMVC, so as not to process the filter preevent. Only post-events are handled:
Object asyncManagerAttr
servletRequest . getAttribute ( WEB_ASYNC_MANAGER_ATTRIBUTE ); return asyncManagerAttr instanceof
WebAsyncManager
; 5. Full link asynchronous
We introduced the above serlvet asynchronous, believes that attentive students see it did not seem to solve the fundamental problem, my IO congestion still exists, just changing the position, when the IO thread pool will also make business calls frequently fill up quickly, although serlvet container thread is not blocked, but the business still can become unavailable.
So how can we solve the above problems? The answer is full link asynchrony. Full link asynchrony pursues no blocking, fills your CPU, and presses the machine’s performance to the extreme. The model is shown as follows:
What exactly does NIO Client do, as shown in the following model:
This is our full link asynchronous diagram (some thread pools can be optimized). The core of full link is that whenever we encounter IO calls, we can use NIO to avoid blocking, which also solves the embarrassing situation that the business thread pool is full.
5.1 Asynchronous Remote Invocation
We usually use RPC or HTTP for remote calls. For RPC, thrift, HTTP,motan, etc. support asynchronous calls. The internal principle is also based on the event-driven NIO model. For HTTP, apachehttpclient and okhttp also provide asynchronous calls. Here’s a quick overview of how Http asynchronous calls work:
public
class
HTTPAsyncClientDemo
{
public
static
void main ( String [] args )
throws
ExecutionException ,
InterruptedException ,
IOReactorException
{
// Specific parameter meanings will be explained later
// Apache provides the ioReactor parameter configuration, here we set the IO thread to 1
IOReactorConfig ioReactorConfig
IOReactorConfig . custom (). setIoThreadCount ( 1 ). build ();
Create an ioReactor based on this configuration
ConnectingIOReactor ioReactor
new
DefaultConnectingIOReactor ( ioReactorConfig );
/ / asyncHttpClient use PoolingNHttpClientConnectionManager manage our client connection
PoolingNHttpClientConnectionManager conManager
new
PoolingNHttpClientConnectionManager ( ioReactor );
// Set the maximum number of connections conmanager.setMaxTotal (100);
// Set the maximum number of connections per route conmanager.setDefaultMaxPerRoute (100);
// Create a Client
CloseableHttpAsyncClient httpclient
HttpAsyncClients . custom (). setConnectionManager ( conManager ). build ();
// Start the client httpclient . start ();
// Execute request
final
HttpGet request1
new
HttpGet ( “www.apache.org/” );
Future < HttpResponse
future
httpclient . execute ( request1 ,
null );
// and wait until a response is received
HttpResponse response1
future . get ();
System . out . println ( request1 . getRequestLine ()
“- >”
response1 . getStatusLine ());
// One most likely would want to use a callback for operation result
final
HttpGet request2
new
HttpGet ( “www.apache.org/” ); httpclient . execute ( request2 ,
new
FutureCallback < HttpResponse
(a)
{
// This method is called back when Complete succeeds
public
void completed ( final
HttpResponse response2 )
{
System . out . println ( request2 . getRequestLine ()
“- >”
response2 . getStatusLine ());
}
public
void failed ( final
Exception ex )
{
System . out . println ( request2 . getRequestLine ()
“- >”
ex );
}
public
void cancelled ()
{
System . out . println ( request2 . getRequestLine ()
” cancelled” );
}
});
}} The entire class diagram of httpAsync is shown below:
So for our HTTPAysncClient we actually ended up using InternalHttpAsyncClient, and inside of InternalHttpAsyncClient we have a ConnectionManager, so this is the manager that we’re managing connections to, And only an implementation that is in in httpAsync PoolingNHttpClientConnectionManager, has two to the connection manager that we are one of the biggest concerns of Reactor, a Cpool.
Reactor: All reactors implement the IOReactor interface. There will be have a Reactor in PoolingNHttpClientConnectionManager, That is DefaultConnectingIOReactor, this DefaultConnectingIOReactor, handle Acceptor. A excutor in DefaultConnectingIOReactor method, generating IOReactor BaseIOReactor in our figure, IO operation. This model is our 1.2.2 model above
A CPool CPool: in PoolingNHttpClientConnectionManager, is mainly responsible for controlling our connection, we said above maxTotal and defaultMaxPerRoute, are controlled by them, If each route is full, it breaks the oldest link, and if the total is full, it stores the header in the Queue, rewiring it as it frees space.
5.2 Database Invocation Asynchronization
For the database call general framework does not provide asynchronous method, here recommend their own packaging or use of open source online, here our company has an open source github.com/ainilife/ze… Asynchrony is well supported
6. The final
Asynchrony is not a silver bullet for high concurrency, but it does improve your machine’s QPS, throughput, and so on. If some of the models mentioned above can be reasonably optimized, and then applied, I believe that can be of great help to your service.