Properly configure the number of threads in the thread pool

preface

With the development of multi-core CPU, multi-threaded programming is more and more widely used. In order to reduce the extra resource cost caused by frequent creation and destruction of threads, thread pool technology is introduced. Using thread pool technology we need to set the number of core threads and the maximum number of threads, many middleware will also set these two parameters by ourselves, so how to set these two parameters scientifically and reasonably? Let’s look at it together.

plan

Generally our programs can be divided into the following types, CPU intensive (computing intensive), IO intensive, hybrid, and each type of program we have different configuration scheme.

1 CPU Intensive

CPU intensive also means computation intensive, often refers to the algorithm complex program, requires a lot of logical processing and calculation, the CPU is always working during this period.

In this case, CPU utilization is very high, so we can set the maximum number of cores to the number of CPU cores.

In this case, the CPU was running at almost full capacity, and the number we configured was not effective at large, but increased the additional switching overhead.

In Java Concurrency in Practice, we recommend setting the maximum NUMBER of CPU-intensive threads to the maximum number of threads = number of CPU cores + 1 for maximum efficiency.

The number of core threads is usually set to number of core threads = maximum number of threads x 20%.

As shown in the figure below, CPU intensive computing tasks are more persistent.

2 IO Intensive

IO intensive means that more of our program’s work is to read data from disk, memory, or network. During I/O, our threads are blocked and the CPU is actually idle, so that our operating system can switch to other threads to use CPU resources.

Usually, remote interface calls, database data acquisition, and buffer data acquisition are IO operations.

The number of threads can be calculated using the following formula: Maximum number of threads = number of CPU cores/(1 – blocking percentage).

We can easily understand that in a certain request, the request duration is 10 seconds, the IO call time is 8 seconds, then our blocking percentage is 80%, the efficient utilization of CPU is 20%, assuming an eight-core CPU, our number of threads is 8 / (1-80%) = 8/0.2 = 40.

This means that our eight-core CPU can run 40 threads at full capacity in this case. In this case, we can set the maximum number of threads to 40 and process other threads while the system performs I/O operations.

Normally we set the maximum number of threads = number of CPU cores * 2.

The number of core threads is usually set to number of core threads = maximum number of threads x 20%.

As shown in the following figure, IO intensive is a task that computations only part of the time and blocks IO for a long time. So the CPU can handle more tasks at the same time.

3 hybrid

Hybrid programs, which include both CPU – intensive and IO – intensive programs.

For such programs, we can divide tasks into CPU intensive tasks and IO intensive tasks, and use different thread pools for each task. This allows us to configure different parameters for the thread pool for both cases.