Java thread stack

  • Linux jstack command
Jstack 7756(Java process number) > java.stack

Copy the code
  • Linux kill command
kill-3 7756(Java process NUMBER)



Send the QUIT signal to the JVM

Copy the code

Java thread

  • JVM creation thread

  • User threads
The main function represents the main thread

Copy the code
  • The first line of the thread stack

1, Java thread conversion LINUX kernel library pThreads are local threads

2. Tid language id, such as Java thread number

3. Nid (Native Thread) Indicates the ID of a Native Thread running on the operating system

Copy the code
  • Thread Call Meaning

Stack information about the call relationship of one of the rows


  • Pstatck 7756(Java process number)
Nid (base 16) == LWP(base 10)

Copy the code

The hexadecimal NID


Base 10 LWP (Lightweight process)


Lock release

  • Wait () releases the watch lock

  • Sleep will occupy the lock

  • When a thread owns a lock
The stack will print -locked <Oxe7402c48>

Copy the code
  • When the thread waits for another thread to release the lock
-waiting to lock <Oxe7402c48>

Copy the code
  • If there is a wait call in your code
Waiting on <Oxe7402c48>

This can lead to deadlocks that are locked in an infinite loop waiting for other locks

Copy the code

The thread deadlock


Thread 0 acquires lock 0 thread 1 acquires lock 1

Thread 0 acquires lock 1 thread 1 acquires lock 0 at time 2

Two threads each acquire a lock and wait for the other lock

Copy the code

The Java stack directly gives the result of a deadlock

Deadlock error


Deadlock details


Deadlock analysis results in CPU overload analysis

Method 1

steps


Why do we get rid of itwaitAnd sleep because it does not consume CPU resources

Copy the code

why


Method 2 TOP stack method (one hit)


JNI is a Java Native Interface

Copy the code

5578 is the Java process number

Prints all threads in this process

CPU consumption in reverse order shit+ H

Jstack prints out a stack of threads with a bunch of threads in it and one of them is equal to the PID that consumes the most CPU

I can directly locate a problem in this line of code

Copy the code

The thread state that does not consume CPU

  • The RUNNABLE state of the JVM
Network IO does not consume CPU and is a synchronous blocking process

Copy the code
  • TIMED_WAITING(on object monitor)
obj.wait(time)

Copy the code
  • TIMED_WAITING(sleeping)
Thread.sleep(time)

Copy the code
  • TIMED_WAITING(parking)
suspended

Copy the code
  • WAITING(on object monitor)
obj.wait()

Wake up with notify()

Copy the code
  • BLOCKED (on object monitor)
Waiting watch lock

Copy the code
  • WAITING(parking)
suspended

Copy the code

Java thread stack

  • Stack information for a thread

  • Multiple thread stack

The performance deteriorates due to insufficient resources

A large number of threads stop in the same invocation context

Copy the code

why


Failed to release the connection in time


Chain analysis caused by multiple locks

Many threads are waiting for different locks and some lock contention may be caused by another lock object contention and need to find the root cause

Copy the code

Case analysis


WebAPI performance bottlenecks

Online performance

1. RECQ(receiving queue)

2. The timeout log is printed crazily

3. Restart the WebAPI process to temporarily relieve the pressure

Copy the code

Solving steps


Thread A


Thread B


Online fault handling mode

  • Keep the site
For example, storing stack snapshot information via jStack

Copy the code
  • Restore service
Services such as restart and reboot are restored immediately

Copy the code
  • Trying to solve
  • validation

Normal operation

  • restart
  • The rollback
  • demotion
  • Picking machine

Symptom collection & Fault location


Restart works if potential bugs are fired

Copy the code

Troubleshooting & Service recovery


Identify service processes. Troubleshoot problems


CPU

top 

shift+p Indicates that pid processes are numbered in ascending order

shift+m In descending order by occupied memory size

Copy the code

memory

free -g

Copy the code

memory

vmstat -n

Copy the code

Number of TCP connections

netstat -aonp|grep tcp |wc -l

Copy the code

Port mapping

netstat -natp|sort -m

Copy the code

The disk is

iostat -d 1

Copy the code

File occupancy

ls -l /proc/*/fd

Copy the code

Target in-service observation


Question assistance methodology


Exception Handling Principles

Mistakes can never be avoided and they can never happen from an anticipated Angle

Maintain the availability of core system functions as much as possible in the event of a failure

Copy the code