“This is the seventh day of my participation in the First Challenge 2022.

Benevolent people see that benevolence, wise people see that wisdom.

At work and in the interview, I will be asked about some tuning content, such as mysql, JVM, tomcat and so on. Because this problem is quite big, I will only talk about it today. It is also a problem that will be encountered in the interview. It is important to note that the CPU usage shown in the top command is aggregate, and the usage of a single CPU is divided by the number of CPU cores to get the result.

The troubleshooting procedure is as follows:

  • 1 Run the top command to find the PROCESS ID PID that consumes the most CPU
  • 2 Locate the ID of the thread that consumes high CPU resources based on its PID
  • 3 Stack the current thread and output all stack information of the previous process
  • 4 Convert the thread ID obtained in step 2 to hexadecimal to obtain the result
  • 5 Locate the stack information based on the thread ID
  • 6 Read the corresponding stack information, locate the code, and troubleshoot the fault

1 Locating a Process

In Linux, the top or ps command is used to locate and search processes. Let’s say we find an application with process number 19505.

# see the top command according to x according to the CPU to sort the top - d 2 # if sure is a Java process, can use the ps command to check the process of ps - aux | grep Java # if think top command display of data is too much, you can use the following command. Before the show just A few lines of top | grep "top -" - 8 ACopy the code

2 Search for thread information

To search for threads in the process, run the ps or top command to view information about threads in process 19505.

Ps -mp 19505 -o THREAD,tid,time top -hp 19505 -d 1 -n 1 # The shell print command converts the thread ID with the thread height to hexadecimal, and assuming we find a thread id 89043, the conversion to hexadecimal is 15bd3. printf "%x\n" tidCopy the code

Find the thread stack

To print thread stack information, you need to use the Jstack command of the JVM. If the information about the current process is displayed, then use the grep command to directly match the information about the search thread. Or store thread stack information as text and then search for it. Run the jstack command to query the stack information

19505 # to check the process of tid = 15 bd3 thread's stack jstack 19505 | grep tid - A 30 # use jstat see gc information and memory usage jstat gc or jstat -- gcutilCopy the code

conclusion

In general, the steps for troubleshooting online CPU spikes have been explained, but few companies allow online troubleshooting, and remove the faulty service from online for operation observation. In practical work, this kind of operation is not common, 99% of problems can be located by log, complex systems also have corresponding link tracking system, this kind of operation is rarely used, only used in the interview, the point is also the proficiency of JVM and Linux operating system.