I have drawn 25 pictures to show how thread pool works and how it works

Good memory is better than bad writing, recording is eternal! This is JavaQ base camp, we sincerely invite you to pay attention.

At the end of the last article “SUCH API gateway query interface optimization, I was forced to”, a friend mentioned that the scene in the article is IO intensive operation, not CPU intensive operation, do not need to use thread pool, I guess the friend may want to express that I/O intensive and long blocking time do not use thread pool solution. I/O intensive can use thread pools if synchronization processing time is controlled or blocking waits are controlled.

Friends who have paid attention to my update frequency will find that there is no new content for several days. There are two reasons. One is that I am really busy recently, and the project is pressed. The other one is I am combing skills mapping, subsequent updates will according to this map, is still ongoing, interested friends under continuous attention and I github:https://github.com/wind7rui/JavaHub, continuously updated! Ok, let’s get started on today’s topic thread pools. Note: below many high energy warning, suggested to collect before reading, to prevent lost!

Why use thread pools

When you talk about multithreading, you’re bound to say use a thread pool. Why use a thread pool? Instead, the question can be reversed: what if you didn’t use a thread pool? When need multi-threaded concurrent execution task, only constant through the new Thread to create threads, each to create a Thread to memory space allocated on the heap, at the same time need to allocate a virtual machine, local method stack, the program counter, such as Thread private memory space, when the Thread object is marked as unavailable accessibility analysis algorithm by GC recycling, Such frequent creation and collection requires significant additional overhead. Furthermore, the JVM’s memory resources are limited. If a large number of thread objects are created in the system, the JVM is likely to throw OutofMemoryErrors directly. There are also many threads competing for the CPU and other performance costs.

Since there are so many problems with not using thread pools, let’s look at the benefits of using thread pools:

• Using a thread pool allows you to reuse threads in the pool without having to create new threads every time, reducing the overhead of creating and destroying threads.

• At the same time, the thread pool has a queue buffer strategy, rejection, mechanism and dynamic management the number of threads, a specific thread pool also has executed, cycle timing functions, of the more important thing is the thread pool thread environment isolation can be realized, such as defining the payment functions related to the thread pool and coupons function related to the thread pool, have a problem when one will not affect the other.

How to construct a thread pool object

ThreadPoolExecutor: AbstractExecutorService abstracts ThreadPoolExecutor, which implements the ExecutorService interface. The ExecutorService inherits the Executor interface, so ThreadPoolExecutor indirectly implements the ExecutorService interface and the Executor interface, as shown in the following diagram.

Generally, the execute method is defined in the Executor interface, and the Submit method is defined in the ExecutorService interface, So we can use the execute method to submit tasks when creating an Executor variable that references an instance of the ThreadPoolExecutor object, and the Submit method when creating an ExecutorService variable. Of course we can create variables of type ThreadPoolExecutor directly using the execute method or submit method.

ThreadPoolExecutor defines seven core properties that are the cornerstone of thread pool implementation.

• CorePoolSize (int) : number of core threads. By default, after a thread pool is created, the number of threads in the pool is zero. When a task arrives, a thread is created to execute the task. When the number of threads in the pool reaches corePoolSize, the incoming task is placed on the task queue. Thread pools keep these threads alive for a long time, even when they are idle. Unless allowCoreThreadTimeOut=true is configured, threads with a core number of threads are no longer guaranteed to live in the thread pool for long periods of time and are destroyed when the idle time exceeds keepAliveTime.

• WorkQueue: blocks a queue for tasks to be executed. Threads fetch tasks from the workQueue. If there are no tasks, the queue blocks. When the number of threads in the thread pool reaches corePoolSize, new tasks are put to the queue. The JDK provides four ready-to-use queue implementations: Array-based bounded queue ArrayBlockingQueue, LinkedBlockingQueue unbounded list based queue, SynchronousQueue with one element, PriorityBlockingQueue. Be sure to set the queue length in practice.

• MaximumPoolSize (int) : specifies the maximum number of threads that can be maintained in the thread pool. Threads that are larger than the core and smaller than the maximum number of threads are destroyed when the keepAliveTime is exceeded. When the blocking queue is full, new threads will be created to execute the task, and the number of threads will not be larger than maximumPoolSize.

• KeepAliveTime (long) : The thread lifetime. If the number of threads exceeds corePoolSize and the idle time exceeds the lifetime, the thread will be destroyed. Unless allowCoreThreadTimeOut=true is configured, threads with a core number of threads are no longer guaranteed to live in the thread pool for long periods of time and are destroyed when the idle time exceeds keepAliveTime.

• TimeUnit Unit: unit of the thread lifetime, for example, timeunit. SECONDS.

• RejectedExecutionHandler: Reject the policy to be adopted when the task queue is full and the number of thread pools reaches maximunPoolSize. ThreadPoolExecutor provides four rejection policies: Throw RejectedExecutionException abnormal AbortPolicy (if not specified the default policy), use the caller’s thread to run the task CallerRunsPolicy, discarding a task to be performed, Then try to execute the current task DiscardOldestPolicy, silently discard and DiscardPolicy without discarding the exception. In the project, you can customize the rejection policy for more user experience.

• ThreadFactory: a factory for creating threads. Although the JDK provides a default implementation of a threadFactory, DefaultThreadFactory, a custom implementation is recommended. This allows you to customize the process of thread creation, such as thread grouping, custom thread names, and so on.

Normally we use the constructor of a class to create its objects, and ThreadPoolExecutor provides four constructors.

You can see that the first three methods all end up calling the last method with the longest argument list, where seven attributes are assigned. If you create a thread pool object, use the ThreadPoolExecutor constructor instead of Executors for the following reasons: * If you create a thread pool object, use the ThreadPoolExecutor constructor instead of Executors.

Hand rolled sample

Now that you know the basic structure of ThreadPoolExecutor, let’s walk through some code to see how to use it. The parameters in the sample code are for use only as part of the rationale.

How thread pools work

To illustrate how thread pools work, I use the following seven diagrams. 1. When a task is submitted using the execute method, if the number of threads in the thread pool is smaller than corePoolSize, a new thread is created to execute the submitted task, even if there are idle threads in the thread pool.

2. When a task is submitted using the execute method, when the number of threads in the thread pool reaches corePoolSize, the newly submitted task is added to the workQueue for execution.

3. When a task is submitted using the execute method and the workQueue is full and maximumPoolSize is greater than corePoolSize, a new thread is created to execute the submitted task.

4. When a thread in the thread pool is idle after completing a task, it attempts to fetch a header task from the workQueue.

5. Submit the task using the execute method. When the number of threads in the thread pool reaches maxmumPoolSize and the workQueue is full, RejectedExecutionHandler will reject the new task.

6. If the number of threads in the thread pool exceeds corePoolSize and allowCoreThreadTimeOut=true is not configured, the threads whose idle time exceeds keepAliveTime are destroyed and the number of threads in the thread pool remains corePoolSize.

Note: The above illustration shows that the idle thread is destroyed, keeping the number of threads to corePoolSize, not the thread in corePoolSize.

7. When allowCoreThreadTimeOut=true, any threads that are idle for longer than keepAliveTime are destroyed.

Underlying implementation principles of thread pools

The implementation of ThreadPoolExecutor is quite complex. Here are some important global constants and methods.

CTL is used to represent the state and number of threads in the thread pool. In ThreadPoolExecutor, 32 bits are used to represent the state of the thread pool and the number of threads in the pool. The first three bits represent the state of the thread pool and the last 29 bits represent the number of threads in the pool. Private Final AtomicInteger CTL = new AtomicInteger(ctlOf(RUNNING, 0)) Initializes the status of the thread pool to RUNNING and the number of thread pools to 0.

COUNT_BITS is equal to integer. SIZE – 3. Integer.SIZE is 32 in the source code, so COUNT_BITS=29. CAPACITY indicates the maximum number of threads allowed by the thread pool. The result is as follows.

RUNNING, SHUTDOWN, STOP, TIDYING, and TERMINATED state of the thread pool are TERMINATED as follows.

The processing power of the thread pool is different when it is in different states.

The transitions between different states of the thread pool and their transitions are shown below.

RunStateOf gets the CTL three bits, which is the state of the thread pool. WorkerCountOf gets the lower 29 bits of CTL, which is the number of threads in the thread pool. CtlOf computes the new value of ctlOf, which is the thread pool state and the number of thread pools.

You may be wondering, “Why all of this?” This is because the following source code analysis will use these basic knowledge points. Normally, we use the Execute method of ThreadPoolExecutor to submit tasks, so start with the execute source code.

To make it easier to understand the source code in the figure above, I have drawn a flow chart.

Now that the basic implementation principle of thread pool is clear, let’s focus on the analysis of thread pool how to perform tasks, how to reuse threads and how to determine the idle time limit of threads. Starting with the execute method, we’ll look directly at the addWorker method called inside it, which creates a new thread to execute the task.

The source code encapsulates threads and tasks into Worker, and then adds Worker to HashSet. After successful addition, the thread is started to execute tasks through the start method of thread object. Let’s look at how w = new Worker(firstTask) is executed in the code above.

Inherited AbstractQueuedSynchronizer Worker, and implements the Runnable interface, see clear this task performed by the Worker in the run method finally, and the run method calls the runWorker method, So the focus is on the runWorker method.

In the runWorker method, a loop is used and getTask method is used to continuously getTask execution from the blocking queue. If the task is not empty, the task will be executed. Here, thread reuse is realized. If the task in the queue is null, the Worker is cleared from the HashSet. Note that this is the collection of idle threads. So when does getTask return NULL? Then look at the getTask source code.

At this point, it is clear how threads in the thread pool perform tasks, how they can be reused, and how to determine when a thread’s idle time limit is exceeded.

Finally, I drew a mind map of how thread pools work. Ps: if not Gao Qingtu platform show that the comments section at the end of the article or comments @ me, in addition, in this paper, the whole graphic has been included in GitHub:https://github.com/wind7rui/JavaHub, subsequent other content will be updated to here, welcome to follow, start.

Talk about practical experience

Create a thread pool using the constructor

If you’re careful, you’ll notice that the full article doesn’t introduce Executors, a tool for creating a thread pool. Yeah, I highly recommend not using it because of the thread pool created by the newFixedThreadPool and newSingleThreadExecutor methods in Executors, The length of the blocking queue LinkedBlockingQueue is integer. MAX_VALUE, which may pile up a large number of tasks, resulting in OOM; The maximum number of threads in the thread pool created by newCachedThreadPool is integer. MAX_VALUE, which creates a large number of threads, resulting in OOM. If you create a thread pool, use the Constructor of ThreadPoolExecutor, so that the user of this thread pool will be more aware of how the parameters of the thread pool are set and run, avoiding hidden problems ahead of time.

Use custom thread factories

Why do you do that? The reason is that, when the project scale gradually expands, the thread pool in each system also increases. When thread execution problems occur, threads created by custom thread factory can set meaningful thread names to quickly track abnormal causes and locate problems efficiently and quickly.

Use a custom rejection policy

Although the JDK provides us with some default rejection policies, we can customize rejection policies to meet specific requirements based on the needs of the project or the user experience.

Thread pool partition isolation

Different services are divided into different thread pools with different execution efficiency. This prevents the usage of the whole thread pool from decreasing or becoming unavailable due to certain exceptions, thus affecting the normal running of the entire system or other systems.

summary

In practice, we often use thread pools, and the requirement is not only how to use them, but also how to understand them. Also, how the thread pool works and how the underlying implementation works are also questions that must be asked in the interview, so it is important to grasp this area.

To be honest, it took a lot of rest time to draw these pictures, so if you’re watching, give me a “like” for my original work!

Learn much, and know little! Friends point [watching] is my biggest motivation to continue to update, we will see you next time!