directory

  • background
  • The basic workings of thread pools
  • Problem analysis in high concurrency scenario of thread pool
  • Performance optimization in high concurrency scenarios with thread pools
  • conclusion

background

Hello everyone, today I will tell you a relatively core technology class knowledge, is Java thread pool in production project high concurrency optimization, probably many brothers have heard of the theory of Java thread pool, know how it works, but have never played with Java thread pool in the project. I’ve never played with Java thread pool optimization in a high-concurrency environment, so today we’re going to take a look at Java thread pool optimization in a production project!

The basic workings of thread pools

Since the thread pool to chat, that at the very least, you have to know a little bit about the basic working principle of Java thread pool, if you want to put the thread pool principle clear, even the analysis to the source of the JDK thread pool level, it may have to separate open to write an article, this is not the subject of this, so this article we’ll look the thread pool is the most simple principle to tell everyone about it.

A thread pool, basically, is a pool that has a bunch of threads in it, and these threads don’t get destroyed, they stay in there forever, and then you can keep submitting tasks to the thread pool, and the thread pool will take them out to do your tasks. After the task completes, the thread does not terminate, but returns to the thread pool to continue waiting, as shown in Figure 1 below:

Figure 1

But this time there will be a key problem, that is the number of threads in thread pool is usually limited, note that here is often, because the real principle of Java thread pool, through customized means actually, there are all kinds of different allows Java thread pool, here is one of the most basic situation, The number of threads in the thread pool is fixed and finite.

So if you were to submit too many tasks to the thread pool at once, and all the threads were busy running their own tasks, what would you say if you wanted to submit another task? Can the task be submitted? ** As shown in Figure 2 below:

Figure 2

Figure 3

Problem analysis in high concurrency scenario of thread pool

So here comes the question. The basics of Java thread pooling and usage have been introduced, but what are some of the problems that arise when you put it into a production project? First of all, one of the biggest problems, is committed to the task in the thread pool, may be to perform a variety of network IO task, for instance, the RPC call other services, or background processing large amounts of data in the db, so the finish is likely to cause a thread to execute a task takes a long time, from a few hundred milliseconds to several seconds, even dozens of seconds, may have it, As shown in Figure 4 below:

Figure 4.

The second problem, which you did not notice in the figure above, is that some tasks are RPC calls, which may take only a few hundred ms, and some tasks are massive data operations, which may take tens of seconds. Therefore, there are many different tasks running in a common thread pool, which makes it uncertain when a thread in the pool can complete a task because the task may be an RPC call or a large amount of data processing.

The third question, there may be some task is in an HTTP request, originally is likely to be in the process of an HTTP request processing, will handle multiple time consuming task in turn, to optimize performance, now need to submit multiple tasks to the thread pool, using multiple threads concurrently multiple tasks, improve the performance of this request, The HTTP request waits for all concurrent tasks to complete before returning a response to the user, as shown in Figure 5 below:

Figure 5

Then this time, the system just for end users to provide c interface has high concurrent access scenario, a large number of HTTP requests, each request to submit multiple tasks to the thread pool concurrency, lead to the few idle thread pool threads run full soon, then a large number of tasks into the queue of the thread pool started waiting in line, as shown in figure 6 as follows:

Figure 6.

Many HTTP requests are queued up in the thread pool, so they can’t run. HTTP requests can’t return responses, giving users the feeling of clicking on the front end of an APP/ web page or something like that. As shown in Figure 7 below:

Figure 7.

Performance optimization in high concurrency scenarios with thread pools

According to the production problem, we need to do the improvement of the first largest, is the variety of tasks from a thread pool, let them don’t affect each other, that is to say, RPC task regularly put a thread pool, timing db large amounts of data processing tasks put another thread pool, A separate thread pool is then placed for HTTP request multi-tasking and concurrent processing, and each member uses its own thread pool and resources without affecting each other, as shown in Figure 8 below:

Figure 8.

As shown in the figure above, we have a thread pool dedicated to handling HTTP requests. The pressure is taken off quickly because HTTP requests are usually in the range of tens of ms to a hundred ms, and the overall speed is very fast. There are no scheduled RPC and scheduled DB access tasks in the thread pool to mess with. Therefore, a dedicated thread pool for HTTP requests can easily and happily handle all HTTP request tasks quickly. Even in high concurrency scenarios, thread resources can be increased through the thread pool to reasonably resist high concurrency pressure.

The other is to run thread pool tasks in the production environment of online systems. We usually develop a unified thread pool monitoring framework within the company or project. All thread pool tasks need to be encapsulated in a Class provided by the thread pool monitoring framework. Then through this Class to achieve the task queuing and running time of two dimensions of monitoring data statistics, as shown in the code below.

// Thread task wrapper class, using the ** decoration design pattern **

Thread task wrapper class, using the decorator design pattern
public class RunnableWrapper implements Runnable {
    
    // The actual thread task to execute
    private Runnable task;
    // The time when the thread task was created
    private long createTime;
    // Start time when a thread task is run by the thread pool
    private long startTime;
    // The end time when a thread task is run by the thread pool
    private long endTime;
    
    // When the task is created, its creation time is set
    // However, it is possible that the task submitted to the thread pool will be queued in the thread pool
    public RunnableWrapper(Runnable task) {
        this.task = task;
        this.createTime = new Date().getTime();
    }
    
    // The run method will not be run while the task is queued in the thread pool
    // However, this method is called when the task has finished queuing and the thread pool is given a chance to run
    // Now you can set the start running time of the thread task
    public void run(a) {
        this.startTime = new Date().getTime();
        
        // You can invoke the API of the monitoring system to report monitoring indicators
        // Use starttime-createTime to queue tasks
        // monitor.report("threadName", "queueWaitTime", startTime-createTime);
        
        // You can then call the run method of the actual wrapped task
        task.run();
        
        // After the task is complete, the end time of the task is set
        this.endTIme = new Date().getTime();
        
        // You can invoke the API of the monitoring system to report monitoring indicators
        // Use endtime-startTime for thread tasks
        // monitor.report("threadName", "taskRunTime", endTime - startTime);}}Copy the code

As you can see from the above code, as long as all tasks submitted to the thread pool are wrapped in a RunnableWrapper class wrapped in decorator mode, we can get the creation time, start time and end time of the thread task. Then the queuing and running time of this task can be calculated and reported through the monitoring system.

At this point, by configuring alarm conditions in the monitoring system, time indicators of each task in different thread pools can be reported. At the same time, if the queuing time or running time of a thread in a thread pool exceeds the configured threshold, an alarm will be automatically generated, as shown in Figure 9 below:

Figure 9.

conclusion

Well, the article here, today will give you our thread pool in the production of the production problems in the project and how to improve the high concurrency, and production environment monitoring plan, all tell you, hope you put this to use, and later in the project use the thread pool, flexible use of both knowledge learned in the article.

END

Scan code for free 600+ pages of Huisei teachers original fine articles summary PDF

Summary of original technical articles