Since the creation and destruction of threads are relatively heavy weight operations for the operating system, the pooling of threads has been practiced in various languages. Of course, in the Java language, the thread pool is also a very important part. With the encapsulation of thread pool by Doug Lea, it is very convenient for us to use. However, it is also possible to misunderstand the configuration parameters of the thread pool because you do not know its implementation.

When you submit a task to a thread pool, as you often see in technical books or blogs, the execution logic of the thread pool looks like this:

  • 1, When a task is submitted, the thread pool first checks whether the number of threads running reaches the core number of threads. If not, a thread is created.
  • 2. If the number of threads running in the thread pool reaches the core number of threads, the task will be placed in the BlockingQueue.
  • 3. If the BlockingQueue is full, the thread pool will attempt to expand the number of threads to the maximum thread pool capacity.
  • 4. If the number of threads in the current thread pool has reached the maximum thread pool capacity, a rejection policy will be executed to reject the task submission. The process is shown below (from the Meituan technology blog) :

The process description is fine, but it can lead to misunderstandings if certain points are left out of consideration, and if the description is too idealistic, some very tricky problems can arise if the configuration is not taken into account in the runtime environment.

Core pool

The part of the thread pool with the number of threads less than or equal to the coreSize is called the core pool. The core pool is the permanent part of the thread pool. Generally, the internal threads will not be destroyed, and most of the tasks we submit should be executed by the threads in the core pool.

Misunderstanding of thread creation timing

One of the most common misconceptions about core pools is not knowing when to create threads in the core pool. For this, I don’t think it’s too much to throw 10% of the blame at Doug Lea. Because he wrote in the document, “If fewer than corePoolSize threads are running, Try to start a new thread with the given command as its first task. Running in our understanding means that the current thread is scheduled by the operating system, has the operating system time sharding, or is understood to be performing a task.

Based on this understanding, it is easy to assume that if the QPS of a task is very low, the number of threads in the thread pool will never reach the coreSize. That is, if we configure the CoreSize to 1000, the QPS is only 1, and the single task takes 1s, then the core pool size will always be 1, and the core pool will only be expanded to 3 even if there is traffic jitter. Because a thread executes one task per second, it’s just enough to handle 1Qps without creating a new thread.

The creation process

But if you simply design a test that prints out the thread stack using jstack and counts the number of threads in the thread pool, you can see that the number of threads in the thread pool increases gradually as the task is committed until it reaches CoreSize. Since the core pool is designed to be a resident pool that can carry daily traffic, it should be initialized as soon as possible, so the logic of the thread pool is that each task will create a new thread before the coreSize is reached. The corresponding source code is as follows:

public void execute(Runnable command) {\ ... \ int c = ctl.get(); \ if (WorkerCountOf (c) < CorePoolSize) {// WorkerCountOf () method is to get the number of threads in the thread pool \ if (addWorker(command, true))\ return; \ c = ctl.get(); \} \... The \}

The running state in the document also means that the thread has been created. We also know that after the thread has been created, it will try to fetch and execute the task from the BlockingQueue in a while loop. It is fair to say that it is running.

For this reason, we do not expect the JVM to do JIT optimization for the hot code when we warm up some high concurrency services. It is important to warm up thread pools, connection pools and local cache.


BlockingQueue is another important component of the thread pool. First, it acts as an intermediary in the producer-consumer model of the thread pool. In addition, it can buffer large bursts of traffic, but understanding and configuring it is often error-free.

Operating model

The most common mistake is not understanding the thread pool runtime model. The first thing to be clear is that the thread pool does not have an accurate scheduling function, i.e. it cannot sense which threads are idle and assign the submitted tasks to the idle threads.

Thread pool uses “producer-consumer” mode, except for the task created by the thread (the firstTask of the thread) does not enter the BlockingQueue, all other tasks must enter the BlockingQueue, waiting for the consumption of threads in the thread pool. Which thread consumes the task depends entirely on the scheduling of the operating system. The corresponding producer source is as follows:

public void execute(Runnable command) {\ ... \ if (IsRunning (c) &&WorkQueue. Offer (command)) {IsRunning () is the state of thread processing \ int reCheck = ctl.get(); \ if (! isRunning(recheck) && remove(command))\ reject(command); \ else if (workerCountOf(recheck) == 0)\ addWorker(null, false); \} \... The \}

The corresponding consumer source is as follows:

private Runnable getTask() {\ for (;;) {\... \ Runnable r = timed ? \ workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :\ workQueue.take(); \ if (r ! = null)\ return r; \... \} \}

Buffering of BlockingQueue

Based on the producer-consumer model, we might assume that if enough consumers are configured, there will be no problem with thread pools. In fact, we have to consider the concurrency factor.

Consider the following: There are 1000 tasks to be submitted to the thread pool for concurrent execution. If the thread pool is initialized, all of them will be placed in the BlockingQueue for consumption. In the extreme case, none of the consuming threads will be executed. If the Size of the configured BlockingQueue is less than 1000, additional requests will be rejected.

So what’s the probability of this limiting case happening? The answer is very large, because the operating system has a very high priority for scheduling I/O threads. We usually start with the preparation or completion of I/O (e.g., Tomcat receives an HTTP request), so it is very likely to be scheduled to Tomcat threads, which are constantly submitting requests to the thread pool. The consumer thread cannot be scheduled, resulting in a stack of requests.

The service I am responsible for has such a situation where the request was rejected unexpectedly. In the QPS 2000 test, the average response time was 20ms. Under normal circumstances, 40 threads can balance the production speed without stacking. However, when the BlockingQueue Size is 50, the request will still be rejected by the thread pool, even if the thread pool coreSize is 1000.

In this case, the important significance of BlockingQueue is that it is a container that can store tasks for a long time, buffering the thread pool at a small cost. The maximum number of tasks submitted simultaneously is called the concurrency value. When configuring the thread pool, it is very important to understand the concurrency value.

The calculation of concurrent quantities

We often use QPS to measure service stress, so we often use this value when configturing thread pool parameters, but sometimes the correlation between QPS and concurrency is not so high, and the QPS should be paired with task execution time to calculate the peak concurrency.

For an interface with exactly the same request spacing and an average QPS of 1000, what is the peak concurrency? There is no way to estimate it, because if the task takes 1ms to execute, its concurrency is only 1; If the task execution time is 1s, then the peak concurrency is 1000.

But if you know the execution time of the task, can you calculate the concurrency? No, because if the interval of the request is different, the request in 1min will be sent in one second, and the concurrency will be multiplied by 60. That is why it says that we know the QPS and the task execution time, and the concurrency can only be calculated.

My general rule of thumb for concurrency is QPS * average response time with twice as much redundancy, but if the business is important, it doesn’t hurt to set the Size of BlockingQueue higher (1000 or more) because of the limited amount of memory used per task.

Consider the runtime


In addition to the cases mentioned above, GC is also an important factor. We all know that GC stops the World, but in this case the World refers to the JVM, while the operating system prepares and completes a request I/O. The JVM stops, but the operating system still accepts the request normally and executes it after the JVM recovers. So GC does stack requests.

The above calculation of concurrency must take into account the fact that the stack of requests will be processed in GC time. The stack of requests can be simply calculated by QPS * GC time, and it is important to remember to allow for redundancy.

Business peak

In addition, be sure to consider business scenarios when configuring thread pool parameters. If most of the interface traffic is coming from a timer, then the average QPS is meaningless and the thread pool design should consider setting a larger value for the Size of the BlockingQueue. If the traffic is very uneven, there is only a small period of time in a day with high traffic, and the thread resources are tight, it is necessary to consider leaving a large redundancy for the thread pool’s MAXSIZE. In cases where traffic spikes are obvious and response times are less sensitive, you can also set up a larger BlockingQueue to allow a certain amount of pile-up of tasks.

Of course, in addition to experience and calculation, regular pressure testing of the service is undoubtedly more helpful to grasp the real situation of the service.


When summarizing the configuration of the thread pool, my biggest feeling is that I must read the source code! Read the source code! Read the source code! You can’t get through some important concepts just by looking at the summaries of some books and articles, and even if you do understand most of them, it’s easy to get stuck in some corners. After a deep understanding of the principle, the ability to flexibly configure in the face of complex situations.

Follow the official account: North Tour Learning Java, reply [721] to receive my sorted thread pool learning materials and selected interview questions.