CPU alarms in the production environment:

13:40 Received our production environment server Green Edition CUP overload alarm notification.

Top discovery processes 30247 and 28337 occupy more than 200 CPU and more than 100 CPU and basically occupy three of the four cores.

Arthas investigation process:

1. Open Arthas to find the corresponding waybill module 30247 and payment module 28337 and select the waybill module to enter:

java -jar arthas-boot.jar
Copy the code

Thread 35 and thread 12042 occupy 49% of the CPU.

dashboard
Copy the code

Thread 35 thread 12042

thread 35
thread 12042
Copy the code

To generate a random string containing at least 2 digits, we use the uniform utility class method, which first randomly generates a 10-digit character pool through uuid.randomuuid (), and then randomly needs a string of digits from the character pool. If the random pool of 10 characters are all letters, then the second random will appear an infinite loop, the problem code is as follows:

public static String getRandomStr(boolean numberFlag, int length) {
    String retStr = "";
    String strTable =
        numberFlag
            ? UUID.randomUUID().toString().replaceAll("-", "").substring(0, 10)
            : "1234567890abcdefghijkmnpqrstuvwxyz";
    int len = strTable.length();
    boolean bDone = true;
    do {
      retStr = "";
      int count = 0;
      for (int i = 0; i < length; i++) {
        double dblR = Math.random() * len;
        int intR = (int) Math.floor(dblR);
        char c = strTable.charAt(intR);
        if (('0' <= c) && (c <= '9')) {
          count++;
        }
        retStr += strTable.charAt(intR);
      }
      if (count >= 2) {
        bDone = false;
      }
    } while (bDone);

    return retStr;
  }
Copy the code

5, offline simulation less than twenty thousand times UUID. RandomUUID () will appear in the first ten full letter situation.

6, the final reason is the CPU high speed caused by the loop, fix the code, increase the judgment of whether all letters, the first random out of the 10 character pool are letters, then random again.

Arthas common commands:

Installation:

The curl – arthas.aliyun.com/arthas-boot O… java -jar arthas-boot.jar

Basic commands

Cat — Prints the contents of the file, similar to the Linux cat command echo — prints parameters, similar to the Linux echo command grep — searches for matches. Similar to the Linux grep tee command, which copies standard input to standard output and the specified file, and the Linux tee command, which returns the current working directory, Similar to the Linux command CLS — clear the current screen area session — view information about the current session reset — reset enhanced classes, restore all Arthas enhanced classes, Arthas server shutdown resets all enhanced classes version — prints Arthas version number loaded by current target Java process history — prints command history quit — exits current Arthas client, Other Arthas clients not affected Stop — close the Arthas server and exit keymap for all Arthas clients — Arthas shortcut list and custom shortcuts

The JVM related

Sysprop — View and modify system properties of JVM sysenv — View environment variables of JVM Vmoption — View and modify option PerfCounter for diagnostics in the JVM — view Perf Counter information for the current JVM logger — View and modify Logger getStatic — view static properties of the class Heapdump dump Java heap, similar to the heapdump function of the jmap command

The class/this correlation

Re-define — Load an external.class file. Define — load an external.class file. Define — load an external. Re-define the JVM dump — dump the byte code of a loaded class into a specific directory classloader — check the classloader inheritance tree, urls, class loading information, use classloader to getResource

Monitor/watch/trace

① The monitor method monitors monitor -c 5 demo.MathGame primeFactors

-c 5 Uncollected statistics period Default 120s ② The range that the watch can observe is: MathGame primeFactors “{params,target,returnObj}” -x 2 -b -s -n 2 -x 2 Depth of attribute traversal for output results -b -s before method call MathGame primeFactors “{params[0],throwExp}” -e -x 2 -e indicates that the internal invocation path of the ③trace method is triggered only when an exception is thrown. And prints the time spent on each node in the method path trace Demo.mathGame Run