preface

With the popularity of micro service, monomer application is split into separate micro process, may be a simple request, need to deal with the multiple micro service, this is actually increase the probability of error, so how to ensure that in a single micro service problems, to minimize the negative impact of the whole system, this requires us to introduce today’s thread.

Threading model

Before introduction thread isolation, we know the first container, mainstream framework threading model, because the service is a separate process, the call is between network IO, network IO handling containers such as tomcat, communication framework such as netty, micro service framework such as dubbo, are very good to help us deal with the underlying network IO streams, Let us pay more attention to business processing;

Netty

Netty is a high performance communication framework based on Java NIO. It uses the master-slave multithreading model, as shown below:


EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup();

ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup);
Copy the code

The main thread is a single thread, and the slave thread is a pool of threads that default to CPU *2. We can do a simple test in our business handler:

    public void channelRead(ChannelHandlerContext ctx, Object msg) {
        System.out.println("thread name=" + Thread.currentThread().getName() + " server receive msg=" + msg);
    }
Copy the code

The server prints the current thread when reading the data:

thread name=nioEventLoopGroup-3-1 server receive msg="..."
Copy the code

You can see that the thread used here is actually the same as the IO thread;

Dubbo

Dubbo’s basic communication framework uses Netty, but Dubbo does not directly use Netty’s IO thread to process business. It can simply output the current thread name on the producer side:

The thread name = DubboServerHandler - 192.168.1.115:20880 - thread - 2,...Copy the code

You can see that the business logic is not used by nioEventLoopGroup threads. This is because Dubbo has its own thread model.


  • allAll messages are dispatched to the thread pool, including requests, responses, connection events, disconnection events, heartbeats, and so on.
  • directNone of the messages are dispatched to the thread pool and are executed directly on the IO thread.
  • messageOnly the request response message is dispatched to the thread pool, other connection disconnection events, heartbeat, and other messages are executed directly on the IO thread.
  • executionOnly request messages are dispatched to the thread pool. No response, response, and other disconnection events, heartbeat, etc., are executed directly on the IO thread.
  • connectionOn an IO thread, disconnection events are queued, executed one by one, and other messages are dispatched to the thread pool.

Dubbo uses FixedThreadPool by default and the number of threads is 200 by default.

Tomcat

Tomcat can be configured with four thread models: BIO, NIO, APR, AIO; Tomcat8 starts with NIO by default. This model is very similar to Netty’s threading model (Reactor model), which I won’t mention here). MaxThreads specifies the number of workers to handle I/OS. The default value is 200. You can print the current thread name in the business Controller:

ThreadName=http-nio-8888-exec-1...
Copy the code

The thread that handles the business is Tomcat’s IO thread.

Why thread isolation

Can know from the above introduction thread model, dealing with business or use IO thread such as Tomcat and netty, it will have what problem, such as the current service process need synchronous calls another three micro service, but because of some service problems, cause the thread block, then block is piling up, fill all IO thread, Eventually the current service cannot accept the data until it crashes; Dubbo isolates THE I/O thread from the service thread. If the problem occurs, the I/O thread will not be affected. However, if the problem occurs, the service thread will be occupied. The purpose of thread isolation is to keep a problem in a small area so that it does not affect the whole world.

How do I do thread isolation

The principle of thread isolation is also very simple. Each request is assigned a separate thread pool, and each request is not affected by each other. Mature frameworks such as Hystrix and Sentinel can also be used.

Thread pool isolation

SpringBoot+Tomcat do a simple isolation test, to facilitate simulation configuration of MaxThreads=5, provide isolation Controller, roughly as follows:

@RequestMapping("/h1")
String home() throws Exception {
    System.out.println("h1-->ThreadName=" + Thread.currentThread().getName());
    Thread.sleep(200000);
    return "h1";
}
    
@RequestMapping("/h3")
String home3() {
    System.out.println("h3-->ThreadName=" + Thread.currentThread().getName());
    return "h3";
}
Copy the code

Request 5 times **/h1 request, request again /h3**, observe log:

h1-->ThreadName=http-nio-8888-exec-1
h1-->ThreadName=http-nio-8888-exec-2
h1-->ThreadName=http-nio-8888-exec-3
h1-->ThreadName=http-nio-8888-exec-4
h1-->ThreadName=http-nio-8888-exec-5
Copy the code

It can be found that h1 requests occupy 5 threads, and Tomcat cannot accept requests when h3 is requested. Modify h1 requests to use thread pools:

ExecutorService executorService = Executors.newFixedThreadPool(2);
List<Future<String>> list = new CopyOnWriteArrayList<Future<String>>();
@RequestMapping("/h2")
String home2() throws Exception {
    Future<String> result = executorService.submit(new Callable<String>() {
        @Override
        public String call() throws Exception {
            System.out.println("h2-->ThreadName=" + Thread.currentThread().getName());
            Thread.sleep(200000);
            return "h2"; }}); list.add(result); // Demote processingif (list.size() >= 3) {
        return "h2-fallback";
    }
    String resultStr = result.get();
    list.remove(result);
    return resultStr;
}
Copy the code

The above part of the pseudocode, using the thread pool to execute asynchronously, and beyond the limit of the degraded processing, so that the next request h3, not affected; Of course, the above code is rudimentary, so we can use a mature isolation framework;

Hystrix

Hystrix provides two isolation strategies: Bulkhead Pattern and semaphore isolation, of which thread pool isolation is the most recommended and commonly used. Hystrix thread pool isolation creates different thread pools for different resources, different service calls occur in different thread pools, can fail quickly in the case of queue, timeout and other blocking conditions, and can provide fallback mechanism. Here’s a simple example:

public class HelloCommand extends HystrixCommand<String> {

    public HelloCommand(String name) {
        super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ThreadPoolTestGroup"))
                .andCommandKey(HystrixCommandKey.Factory.asKey("testCommandKey")) .andThreadPoolKey(HystrixThreadPoolKey.Factory.asKey(name)) .andCommandPropertiesDefaults( HystrixCommandProperties.Setter().withExecutionTimeoutInMilliseconds(20000)) AndThreadPoolPropertiesDefaults (HystrixThreadPoolProperties. Setter (.) withMaxQueueSize queue size. (5) / / configuration withCoreSize / / (2) Configure the number of threads in the thread pool)); } @Override protected String run() throws InterruptedException { StringBuffer sb = new StringBuffer("Thread name=" + Thread.currentThread().getName() + ",");
        Thread.sleep(2000);
        return sb.append(System.currentTimeMillis()).toString();
    }

    @Override
    protected String getFallback() {
        return "Thread name=" + Thread.currentThread().getName() + ",fallback order";
    }

    public static void main(String[] args) throws InterruptedException, ExecutionException {
        List<Future<String>> list = new ArrayList<>();
        System.out.println("Thread name=" + Thread.currentThread().getName());
        for (int i = 0; i < 8; i++) {
            Future<String> future = new HelloCommand("hystrix-order").queue();
            list.add(future);
        }
        for(Future<String> future : list) { System.out.println(future.get()); } Thread.sleep(1000000); }}Copy the code

Set the number of threads to be queued as 2 and specify the maximum number of threads that can be queued when they are full.

Thread name=main Thread name=hystrix-hystrix-order-1,1589776137342 Thread name=hystrix-hystrix-order-2,1589776137342 Thread name= hystrix-hystrix-orDER-1,1589776139343 Thread name= hystrix-hystrix-ORder-2,1589776139343 Thread Name = hystrix hystrix - order - 1158776413-43 Thread name = 43 Thread hystrix - hystrix - order - 2158776413 Name = hystrix hystrix - order - 2158776433-43 Thread name = the main, fallback for the orderCopy the code

The main thread execution can be understood as IO thread, and hystrix thread is used for business execution. The number of threads 2+ queue 5 can process 7 concurrent requests at the same time, and the excess part will directly fallback.

Semaphore isolation

The advantage of thread pool isolation is that the isolation degree is relatively high, and the thread pool of a resource can be processed without affecting other resources. However, the cost is that the overhead of thread context switch is relatively high, especially for the low latency call has a relatively large impact. In the introduction of thread model above, we found that Tomcat provides 200 IO threads by default, and Dubbo provides 200 business threads by default. The number of threads is already very large. If each command uses a thread pool, the number of threads will be very large, and the impact on the system will be very large. A lighter form of isolation is semaphore isolation, which limits the number of concurrent calls to a resource rather than explicitly creating a thread pool, so it is less expensive; Hystrix and Sentinel both provide semaphore isolation. Hystrix has stopped updating, while Sentinel does not provide thread isolation at all, or thread isolation is unnecessary and can be replaced with a lighter semaphore isolation.

conclusion

This article from the beginning of the thread model, speak to the IO thread, and why you should separate the IO thread and thread, specific how to realize, finally introduced the more simple light signal isolation, why said more lightweight, business or in IO thread processing, just can limit the number of concurrent of a resource, no redundant thread; This is not to say that thread isolation is not valuable, but it really depends on the situation, on the container you are using, on the thread model of the framework itself