preface

The other day I was looking through all the money I had saved over the years and I felt a sudden sense of satisfaction. However, considering the difficulties of finding information over the years, I decided to sort out these documents and share them with everyone. Notes flashy, which may also have incorrect places, welcome correction. Thank you for your dedication. The document is divided into several parts:Copy the code
  • Source Git

In addition, there will be other notes to share processing, thanks for your support.

1. Basics

1. 1 Common Memory Overflow

> Heap overflow > Metadata overflow, metadata area memory overflow > Direct memory overflow > VM stack and local method stack overflow > Runtime constant pool overflow > method area memory overflowCopy the code

1. 2 Common Causes of Memory Overflow

Copy the code

1. 3 System Thread division

> Serial collector: use a single thread to process all garbage collection work, high efficiency: the data volume is relatively small (100M or so); Single-processor applications with no requirement on response time > Parallel collector: medium and large applications with multiple cpus and no requirement on response time > Concurrent processor: : Medium and large applications with multiple cpus and high requirement on response timeCopy the code

1.4 Four reference types in Java

Strong reference Most of the references we use are actually strong references. This is the most common reference to use, a SoftReferenceCopy the code

1 . 5 TLAB

> Pointer collision - objects refer to the actual memory space, and the pointer to the corresponding pointer, in the heap memory, a piece of memory is a pointer to one2The left is the allocated memory space, and the right is empty. Each time a new object is created, the pointer moves to the right by an object size. This is called a pointer collision, but when multiple threads have high concurrency, there will be a situation where the pointer cannot be modified without time. The full name of TLAB is Thread Local Allocation Buffer, which is a memory Allocation area dedicated to threads. Set up a virtual machine parameters - XX: if UseTLAB, when the thread is initialized, it will also apply for a specified size of memory, only to the current thread to use, so that each thread has a space alone, if you need to allocate memory, on your own space allocation, so there is no competition, can greatly improve the allocative efficiency. The memory of the TLAB space is very small. By default, only the entire Eden space is occupied1%, also can through the options - XX: TLABWasteTargetPercent set TLAB space occupied the percentage of Eden space size. The essence of TLAB is actually three pointer managed areas: Start, top, and end: Each thread allocates a space, say 100KB, from Eden as its own TLAB. Start and end are placeholdees that identify the area of Eden managed by the TLAB. Blocking a chunk of space in Eden from other threads. TLAB only allows each thread to have a private allocation pointer, but the memory space that holds the object is still accessible to all threads, but other threads cannot allocate in this area.// Defect of TLAB
1, the size of TLAB space is fixed, but for a large object at this time, the remaining TLAB space can no longer accommodate it. (for example, a 100KB TLAB comes in with a 110KB object)2, the TLAB space is still a little unused, a little reluctant.3When the Eden space is enough, it is ok for you to apply for TLAB again, but I am not enough. The Eden area in the Heap needs to start GC.4TLAB allows waste of space, resulting in discontinuous space in Eden Zone. I'm gonna need some help in the future.Copy the code

2. Vm

2. 1 Java VM

The Java Virtual Machine (JVM) is a virtual machine process that can execute Java bytecode. It allows Java queries to be used on multiple, arbitrary platforms, but cross-platform is Java programs (including bytecode files), not JVM > classloaders, which will be required at JVM startup or class runtimeclassLoaded into theJVMIn the. > Memory area, the memory is divided into several areas to simulate the storage, recording and scheduling function modules on the actual machine, such as the various function registers on the actual machine orPCA pointer recorder, etc. > The execution engine, whose job is to executeclassThe file contains bytecode instructions that are equivalent to those on the actual machineCPU. > Local method call, callCCThe code for the native method implemented by ++ returns the result. // Runtime data area > program counter:JavaThread private, similar to operating systemPCCounter, which can be viewed as an indicator of the line number of bytecode executed by the current thread. - If the thread is executing aJavaMethod, this counter records the address of the vm bytecode instruction being executed; If what is being executed isNativeMethod, the counter value is null (Undefined) > VM stack (stack memory) :JavaThread private, the virtual machine stack describesJavaMethod execution memory model > local method stack: andJavaThe functions of the VIRTUAL machine stack are similar except that this area isJVMProvide useNativeMethod services > Heap memory (shared by threads) : An area shared by all threads, the primary area managed by the garbage collector - each method executes, creating a stack frame to store local variables, operands, dynamic links, method exits, and so on. - Each method call means that a stack frame is pushed into and out of the virtual machine stack. > Method area (thread shared) : an area shared by each thread to store virtual machine loaded class information, constants, static variables, real-time compiler compiled code, and other data-thread shared area, so this is a thread unsafe area. - Method areas are also a possibilityOutOfMemoryErrorThe area. - The method area stores fromClassFile loads in static variables, class information, constant pools, and code compiled by the compiler.Copy the code

2. 2 Memory heap details


// Memory heap features- It's usnewDoes not hold basic types and object references. - Due to the large number of objects created, the garbage collector mainly works in this area. - The thread shared area is therefore thread unsafe. - Memory overflows can occur, including OutOfMemoryError and StackOverflowError./ / a generationalThe Java heap can also be divided into the New generation and the old generation, and the new generation can be further divided into Eden and Survivor1Area, Survivor2area// Note the ratio:
 8:1:1 + 2:3    
Copy the code

2. 3 Memory stack details

- Thread private area. Each thread has its own virtual machine stack, so this is a thread safe area. - Stores references to basic data types and objects. - Each method creates a stack frame in the virtual machine stack, which is destroyed after the method is executed. - Method stack frame is in the first - out mode of the virtual machine stack. Each stack frame can be divided into local variable tables, operand stacks, dynamic links, method exits, and additional additional information. - There are two possible exceptions in this area: if a thread requests a stack depth greater than allowed by the virtual machine, a StackOverflowError is thrown (usually recursively); An OutOfMemoryError is thrown when the JVM is dynamically expanding and cannot obtain enough memory.Copy the code

2. 4Java memory heap and stack difference

Stack memory is used to store variables of basic types and reference variables for objects; The heap memory is used to store objects in Java, whether they are member variables, local variables, or class variables. The objects they point to are stored in the heap memory. Stack memory belongs to a single thread, each thread will have a stack memory, its stored variables can only be visible in its owning thread, that is, the stack memory can be understood as the thread's private memory; Objects in the heap memory are visible to all threads. Objects in the heap memory can be accessed by all threads. If there is no available space stack memory storage method calls and local variables, the JVM throws Java. Lang. StackOverFlowError error; If the heap memory is no space available to store the generated objects, the JVM throws Java. Lang. An OutOfMemoryError. The stack memory is much smaller than the heap memory, and if you use recursion, your stack will quickly fill up. The -xSS option sets the size of the stack memory, and the -xMS option sets the start size of the heap.Copy the code

2. 6 HotSpot VIRTUAL machine

The HotSpot virtual machine is physically divided into two parts: > Young Generation: Most newly created objects are allocated here: The process by which objects disappear from this region is called "minor GC". Two Survivor Spaces created in Eden -- first GC --> One Survivor space -- keep piling up --> saturate and move to the second Survivor space --> empty the saturated space --> put the remaining into old age > Old age after a few rounds Generation) : Objects that do not become unreachable and survive from the new generation will be copied to this: The process by which objects disappear from the old generation, we call this "major GC" (or "full GC") > card table: A write barrier, 512 bytes, is used to record the use of the old generation to the new generation: Save class constants and string constants > Speed up cache allocation: bump-the-pointer - Track the last object created in the Garden of Eden space, place it at the top, and find that object next time you create it: TLABs (Thread-Local Allocation Buffers) - This scheme allocates each Thread an exclusive chunk of Eden space, so that each Thread accesses only their own TLAB space. In combination with the Bump-the-Pointer technology, you can allocate memory without locking itCopy the code

3. Garbage removal

3.1 Causes of garbage collection

> Programmers cannot automate system GC. GC is typically created in an environment where most objects quickly become unreachable and there are only a few references from old objects to new objectsCopy the code

3.2 Concepts in garbage collection

// stop-the-world: stop-the-world occurs in any GC algorithm: stop-the-world means that the JVM has stopped the execution of the application in order to perform GC: When a stop-the-world occurs, all threads except those needed for GC are in a wait state until the GC task is complete: GC optimization is often about reducing the time that a stop-the-world occurs// Recycle by generation> Why is garbage collection generational: : Different object lifecycles are different, different collection methods can improve the recovery rate > Generational: : young generation: old generation: persistent generation// Cenozoic GC and old GCNew Generation: One Eden zone + two Survivor zones Old Age: The default value of the ratio of Young to Old is1:2, default Eden:from:to=8:1:1YoungGC (MinorGC/YoungGC) : indicates a garbage collection action that occurs in the new generation. Since most Java objects are born and die soon, MinorGC is very frequent and the collection speed is relatively fast. Old GC (MajorGC/FullGC) : Exploiture refers to a GC that occurs in the old days. MajorGC is used to be accompanied by a MinorGC at least once. Be insane. MajorGC is generally slower than MinorGC10Or more times.// Triggers a generational collection modeExploiture GC and Full GC. Scavenge GC: A new object is generated, and the application for a space in Eden fails// Garbage collectorNew Generation collector -Serial collector - ParNew collector? - ParNew collector, a multithreaded version of the Serial collector. Be insane. Parallel Scavenge collector. Serial Old collector? - Serial Old collector: indicates the Old version of the Serial collector. - Parallel Old collector? The Parallel Old collector is an Old version of the Parallel Scavenge. - CMS collector New generation collector + Old collector - G1 collector - ZGC collector// The difference between G1 and CMS• CMS: Concurrent flag clearance. The main steps are: initial collection, concurrent marking, re-marking, concurrent clearing (deletion), and reset. • G1: Main steps: initial marking, concurrent marking, re-marking, copy-clean (collation) • THE disadvantage of CMS is that it is quite CPU demanding. G1 divides memory into multiple chunks, so there is a big requirement on the size of the internal segments. • CMS is cleared, so there will be a lot of memory fragmentation. G1 is defragmenting, so the debris space is small. • Both G1 and CMS are response first, and their goal is to control STW time as much as possible. • The Full GC of G1 and CMS is a single-threaded Mark Sweep Compact algorithm that was optimized for parallel until JDK10.Copy the code
The collector Serial, parallel or concurrent The new generation/the old algorithm The target Applicable scenario
Serial serial The new generation Replication algorithm Response speed priority Client mode in a single-CPU environment
Serial Old serial The old s Tag-finish Response speed priority Client mode and CMS backup plan in a single-CPU environment
ParNew parallel The new generation Replication algorithm Response speed priority Work with the CMS in Server mode in a multi-CPU environment
Parallel Scavenge parallel The new generation Replication algorithm Throughput priority Tasks that operate in the background without requiring much interaction
Parallel Old parallel The old s Tag-finish Throughput priority Tasks that operate in the background without requiring much interaction
CMS concurrent The old s Mark-clear Response speed priority Centralized Java applications on Internet sites or B/S system servers
G1 concurrent both Tag-collate + copy algorithm Response speed priority Server oriented applications, replace CMS in the future

3. 3 Classification of Garbage Collection

    
    
Copy the code

3.4 Common Methods of garbage Collection

// Method 1: Call the system GC method. The developer manually calls the command to trigger the GC
System.gc()
    
// Method 2: Call runtime.geTruntime ().gc(), which will actually invoke System.gc ().
 Runtime.getRuntime().gc()
    
// Mode 3: Use jmap to force GC. The jmap command is used to execute GC
// This command is not foolproof, and an exception may occur if the JVM is occupied and the GC cannot be executed
jmap -histo:live 7544

// Method 4: Use the Jcmd command to execute GC
// The JVM diagnostic command triggers the GC through the Java Diagnostic Command (JCMD)
jcmd 7544 GC.run
    
// Mode 5: Use JConsole or Java Mission Control

Copy the code

3. 5 Garbage collection algorithm

> Application count: there are references/removals to an object. That is, add/delete the number, garbage collection will recycle zero objects > Tag clearance: The first phase marks all referenced objects starting at the root of the reference: The second phase traverses the heap and clears untagged objects > Copying: The memory space of the algorithm is divided into two equal parts. When recycling, the current region is traversed and the used objects are copied to another region. > Mark-compact: : The first stage marks all referenced objects from the root node: The second phase traverses the heap, removing unmarked objects and "squeezing" the living objects into one chunk of the heap, in order to place themCopy the code

6 Common Garbage collectors

3 .6 .1 Serial collector

The Serial collector, the most basic and oldest, pauses all worker threads during garbage collection until the garbage collection process is complete. Here is a schematic of the Serial garbage collector in action:Copy the code

3 .6 .2 ParNew collector

The ParNew garbage collector is actually a multi-threaded version of the Serial garbage collector, which allows the ParNew garbage collector to use multiple threads for garbage collection.Copy the code

3 .6 .3 Parallel Scavenge

Copy the code

3. 6 .4 Serial Old collector

The Serial Old collector is an older version of the Serial collector. The garbage collector works the same way as the Serial collector.Copy the code

3 .6 .5 Parallel Old collector

The Parallel Old collector is an Old version of the Parallel Scavenge. It supports multi-threaded concurrent collections.Copy the code

3 .6 .6 CMS collector

The operation process of CMS garbage collector is relatively complex compared with The previous several garbage collectors. The whole process can be divided into four parts: Initial marking: it needs to Stop The World. Here, only The objects that can be directly associated with GC Roots are marked, so The speed is fast. Concurrency markup: Traversing the entire chain of references to GC Roots from the associated object takes the longest time, but can be run concurrently with the user thread. Reschedule: Fixed The concurrency time, because user threads could cause The markup to change, also need to Stop The World. Concurrent clear: Clear dead objects.Copy the code

3 .6 .7 Garbage First

Garbage First (G1The collector is a landmark achievement in the history of garbage collectors and is primarily oriented towards server-side applications. In addition, G1The collector retains the concept of the Cenozoic and the old age, but the Cenozoic and the old age are not fixed. They are all dynamic collections of a series of regions. • Parallelization and concurrency: G1 can take advantage of The hardware advantages of a multi-CPU, multi-core environment. Using multiple cpus to shorten stop-the-world pauses, The G1 collector can allow Java programs to continue executing concurrently while other collectors would otherwise have stopped Java threads. • Generational collection: While G1 can manage the entire GC heap independently without the need for other collectors, it can handle newly created objects and existing objects that have survived multiple GCS in a different way to get better collection results. • Spatial integration: G1 is a collector based on the "mark-collation" algorithm as a whole, and locally (between two regions) based on the "copy" algorithm, which means that no memory space fragmentation is generated during G1 operation, and a neat amount of available memory is provided after collection. • Predictable pauses: This is another big advantage of G1 over CMS.Copy the code

4. Creation of objects

4. 1 Creation Process

1) detects whether the class is loaded when the virtual opportunity arrivesnewDirective is first checked to see if the parameter of the directive can locate a symbolic reference to a class in the constant pool and whether the class represented by the symbolic reference has been loaded, parsed, and initialized. If not, the class loading process is performed.2Allocate memory for an object After the class is loaded, the VM allocates memory for the object. At this point, the required memory size is determined. Just allocate as much memory as you need on the heap. There are two ways to allocate memory: in the first case, the memory space is absolutely regular and in the second case, the memory space is discontinuous. - It is relatively easy to move the pointer between the occupied memory and the available space. This method is called "pointer collision". - In the case of irregular memory, which is a little more complicated, the VM needs to maintain a list of what memory is available. When allocating memory, you need to find a available memory space, and then record the allocated memory space on the list, which is called the "free list". When multiple threads are concurrent, object A is allocated memory, and object B is allocated memory using the same pointer before it can change the pointer, so there is A problem. There are two solutions to this problem: • The first is to adopt a synchronous approach, using CAS to ensure atomicity of the operation. • In other words, each Thread allocates memory in its own space, that is, each Thread allocates a small chunk of memory in the heap in advance, called Thread Local Allocation Buffer (TLAB), when allocating memory on TLAB, do not disturb each other. This can be determined by the -xx :+/ -Usetlab parameter.3After the memory allocation of an object is complete, the memory space of the object needs to be initialized to zero. This ensures that the object can be used directly even if it has no initial value.4After allocating the memory space and initializing the zero value, the VM needs to set other necessary Settings for the object. The Settings are in the object header, including the class to which the object belongs, the metadata information about the class, the hashcode of the object, the age of GC generation, and so on.5In the virtual machine, the object is created successfully by executing the init method, but in the Java program, the object is only initialized to zero, and there is no actual value assigned by the program code. After the init method is called, the object is actually usable.Copy the code

4. 2 Memory Layout

The memory layout of an object consists of three parts: - Object header: The object header contains two parts of information. • The first part stores the runtime data of the object itself, such as hash codes, GC generational ages, lock status flags, thread-held locks, and so on. • The second part is the type pointer, which is a pointer to the class metadata. - Instance data: data. - Aligned padding: Not necessarily there, but for alignment.Copy the code

4. 3 Object Access Location

Handle location: The Java heap draws a block of memory as a handle pool. The reference contains the address of the object's handle, and the handle contains the address of the object's instance data and type data. Direct pointer access: When a Java heap object is not centered, it must consider how to place the information related to the access type data, whereas reference directly stores the object address.Copy the code

4. 4 Object death

Reference counting algorithm: Adds a reference counter to an object, which is added whenever the object is referenced in a place1; The counter is subtracted whenever an object reference invalidates1. But the counter is0The object is not referenced. Reachability analysis algorithm: Starting with a series of root nodes called "GC Roots", search along the chain of references, and no objects on the chain of references will be reclaimed.// GC Root object: Root object of reachabilityReference objects in the Java virtual machine stack, parameters, local variables, temporary variables, and so on. An object referenced by a class static property in the method area, such as a static variable of a reference type. Object referenced by a constant in the method area. Object referenced in the local method stack. References within the Java virtual machine, Class objects corresponding to basic data types, and some resident exception objects. Synchronizing lock (synchronized) objects held by.Copy the code

4. 6 Class loaders

// What is a classloaderA ClassLoader is used to load Java classes to the Java VIRTUAL Machine. Generally speaking, the Java virtual Machine uses Java classes in the following ways: Java source programs (.java files) are converted to Java bytecode (.class files) after being compiled by the Java compiler, and the classloader is responsible for reading the Java bytecode and converting it into an instance of the Java.lang.class class// The period in which it occurred1Encountered,newGetstatic, putStatic, invokeStatic, if the class has not been initialized, it needs to trigger the initialization first. •2When a class is called to reflect using the java.lang.Reflect package's methods, the class needs to be initialized first if it has not already been initialized. •3When initializing a class, if the parent class has not been initialized, trigger the initialization of the parent class first. •4When the VM starts, the user needs to specify a main class to execute by calling its #main(String[] args) method. The VM will initialize the main class first. •5, when using JDK7 dynamic language support, if a Java lang. Invoke. The final analytical results for MethodHandle instance REF_getStatic, REF_putStatic, REF_invokeStatic method handles, If the class to which the method handle corresponds has not been initialized, it needs to be initialized first.// How to load the Class• The first stage, Loading, is to locate the.class file and load the bytecode contained in the file into memory. • The second stage, Linking, can be divided into three steps, respectively bytecode verification, Class data structure analysis and corresponding memory allocation, and finally symbol table parsing. • In the third stage, Initialization(static attributes and Initialization assignments in classes), Using(execution of static blocks), and so on.Copy the code

4. 7 Detailed Description of ClassLoader

// Java has three class loaders
1.%JRE_HOME%\lib rt.jar, resources.jar, charsets.jar andclassAnd so on. Another thing to note is that it can be startedjvmSpecify -XbootclasspathAnd the path to changeBootstrap ClassLoaderThe load directory of. Such asjava -Xbootclasspath/a:pathThe specified file is appended to the defaultbootstrapIn the path. We can go to my computer and look in the directory above and have a look at thesejarWhether the package exists in this directory. 2.Extention ClassLoaderExtended class loader, loading directory %JRE_HOME% \lib\extIn the directoryjarPackage andclassFile. You can also load -D java.ext.dirsOption specifies the directory. 3.AppClassLoaderTo load the current applicationclasspathAll classes of //classLoadLoading processJavaBased on theLauncherEntry application -LauncherInitialize theExtClassLoaderandAppClassLoader- // Knowledge 1 parent loader is not parent 2Bootstrap ClassLoaderIs made up ofC/C++ write // common methods - to get the parent loader:cl.getParent(),cl.getParent().getParent() - Loaded by the specified fully qualified class nameclass : loadClass(a)Copy the code

// Parent delegation
1First of all, judge thisclassCheck whether the load is successful. 2 WhenclassIf no, go to the root node to check whether the upper layer loader has been loaded (if a layer has been loaded, it will return directly). 3 If yesBootstrap classloaderIs still not loadedBootstrap classloaderTo the specified path to find, if not, then the child loader continues to its corresponding path to find 4, still not found, then return the exception // processTODO: // Consider: loading an object from the top down, finding an object from the bottom upClassloader, using the parent delegate to avoid not knowing whereclassLoaderTo avoid duplicate loading problemsCopy the code

4. 8 Class files

// Class loading process
1.The.java file is compiled to generate aclassFile 2.classloaderThis was first found by the relevant rulesclass3. Then it readsclassHeader file, including the following dataa. 0xCAFEBABE: Checks whether the value isJavacompileb50, 0: Indicates version 4.String.ArrayListThere are different levels ofloaderLoad, the top one is calledBootstrap Classloader", the next stepExtension ClassloaderAnd the bottomApp Classloade5. Then,classIt's going to be loaded into the method area, in the heapnewOut of theclassClassclassLoaded or not 6. EachclassThere will be a local variable area, and a stack of operands, the thread will execute according to the process, such as fetching data from the local variable area, put it on the stack, and finally run it into a number and then put it back into 7. There may be multiple threads, but they work only at the top (in the case of multithreading). Each table is called a stack frame, and multiple tables are the result of method calls //JavaObject headGCGenerational information, lock information, hash code, pointingClassPointer to class meta informationHotspotA VM object header contains two parts of data:Mark Word(Tag field),Klass Pointer(Type pointer) -Klass PointIs a pointer that an object points to its class metadata, which the VIRTUAL machine uses to determine which class the object is an instance of -Mark WordUsed to store the runtime data of the object itself, such as hash codes (HashCode),GCGenerational age, lock status flag, thread-held locks, thread-biasedID, biased timestamp, and so on.JavaObject headers typically occupy two machine codes (in a 32-bit virtual machine, one machine code equals 4 bytes, or 32bits). But if the object is an array type, three machine codes are required becauseJVMThe VIRTUAL machine can passJavaObject metadata informationJavaObject size. You cannot determine the size of an array from the array metadata, so use a block to record the length of the array. //JavaObject instance data The instance data part is the valid information that the object actually stores //JavaObject alignment padding The VM specification requires that the object size must be an integer multiple of 8 bytesCopy the code

Java object header storage structure 32-bit TODO: to be perfected

Good article recommended @ in-depth understanding of Java bytecode structure from a class file _ April Grape’s blog -CSDN blog _Java bytecode

How to view bytecode

// Method 1: Java basic utility class- View the basic bytecode: javap jav.lang. Object - View the basic member: javap -p - View details: javap -v - Disassemble the entire class: Javap -c// Method 2: Query using ASM
<dependency>
    <groupId>org.ow2.asm</groupId>
    <artifactId>asm</artifactId>
    <version>8.01.</version>
</dependency>
<dependency>
    <groupId>org.ow2.asm</groupId>
    <artifactId>asm-util</artifactId>
    <version>8.01.</version>
</dependency>

try {
    ClassReader reader = new ClassReader("java.lang.Object");
    StringWriter sw = new StringWriter();
    TraceClassVisitor tcv = new TraceClassVisitor(new PrintWriter(System.out));
    reader.accept(tcv, 0);
} catch (IOException e) {
    e.printStackTrace();
}

// Method 3: BCEL
<dependency>
    <groupId>org.apache.bcel</groupId>
    <artifactId>bcel</artifactId>
    <version>6.5. 0</version>
</dependency>

try { 
    JavaClass objectClazz = Repository.lookupClass("java.lang.Object");
    System.out.println(objectClazz.toString());
} catch (ClassNotFoundException e) { 
    e.printStackTrace(); 
}

// Method 4: Javassist
<dependency>
    <groupId>org.javassist</groupId>
    <artifactId>javassist</artifactId>
    <version>3.27. 0-GA</version>
</dependency>

try {
    ClassPool cp = ClassPool.getDefault();
    ClassFile cf = cp.get("java.lang.Object").getClassFile();
    cf.write(new DataOutputStream(new FileOutputStream("Object.class")));
} catch (NotFoundException e) {
    e.printStackTrace();
}

// Method 5: Jclasslib

Copy the code

4. 9 Representation of objects in the JVM — OOP-klass

HotSpot uses the OOP-Klass model to represent an Object in the virtual machine, where OOP stands for Ordinary Object Pointer, which represents the instance information of an Object that looks like a Pointer but is actually an Object hidden ina Pointer. Klass, on the other hand, contains metadata and method information to describe Java classes. What it does: To avoid having a vtable in every object, we split the object model into KLass and OOP, where OOP does not have any vtables, and Klass has a vtable that can be used for method dispatch. Klass: a c++ peer of a Java class in HotSpot. It describes Java classes that are created during the load process - implementing the language level Java classes - implementing the distribution of Java objects OOP: while the Java program is runningnewObject, containing the following parts - instanceOopDesc, also known as the object header - Mark Word, which stores the object's run-time record information such as hashcode, GC generational age, lock status flag, thread ID, timestamp, etc. - metadata pointer, That is, pointing to the method area instanceKlass instance-instance dataCopy the code

5. GC monitoring

5.1 When is GC monitored

For example: > when a new generation is moved to an old generation, and how long it takes to do it > When stop the world happens, and how long it takes to do it > GC access interfaces: GUI/CUI CUI GC monitoring methods use separate Jstat cUI applications: cUI or select JVM parameters at startup. Verbosegc: GUI GC is done by a separate graphical interface: Jconsole, JVisualVM, Visual GC Jstat: See appendix for parameter names - VerboseGC: Specified when starting a Java applicationCopy the code

2 Common GC Monitoring Tools

• JPS: JVM Process Status Tool displays all HotSpot virtual machine processes in the specified system. -q: Ignores the class name, Jar name, and parameters passed to the main method, and only prints pid. -m: Outputs the arguments passed to the main method, or if the JVM is embeddednull. -v: Outputs the arguments passed to the JVM through the marked files (.hotspotrc, or the files specified by the -xx :Flags= argument). -j is used to pass JVM options to the Java loader called by Javac, • jstat: JVM Statistics Monitoring is a command that monitors the status of a virtual machine while it is running. It displays running data such as class load, memory, garbage collection, and JIT compilation of a virtual machine process. Common usages include loading and unloading of classes, viewing capacity and usage of new generation, old generation, and persistent generation, viewing garbage collection of new generation, old generation, and persistent generation, Check the capacity and allocation of the Eden and Survior zones in the new generation. • Jinfo: JVM Configuration info. This command is used to view and adjust vm running parameters in real time. • Jmap: The JVM Memory Map command is used to generate heap dump files. • Jhat: JVM Heap Analysis Tool. This command is used together with jmap to analyze dump files generated by Jmap. Jhat has a small BUILT-IN HTTP/HTML server that generates dump analysis results and can be viewed in a browser. • JStack: Java Stack Trace, used to generate a snapshot of the current time in the Java VIRTUAL machine. • HSDIS: JIT generated code decompilation/ / Java cabin• JConsole: The Java Monitoring and Management Console is a built-in Java Monitoring and Management Console that starts with Java5 and is used to monitor memory, threads, and classes in the JVM. • VisualVM: All-in-one troubleshooting tool The JDK comes with an all-in-one tool that can analyze memory snapshots, thread snapshots, monitor memory changes, and GC changes. In particular, the BTrace plug-in, a dynamic trace analysis tool./ / other• [GChisto](GC log Analysis tool -- GChisto) : a professional GC log analysis tool.// JMC : Java Mission Control -> Complete GRAPHICAL interface -> Provide object viewingCopy the code

5.3 Common Monitoring Commands

// Get the memory used by the Java programThe Runtime#freeMemory() method returns the number of bytes of free space. The Runtime#totalMemory() method, the number of bytes of totalMemory. The Runtime#maxMemory() method returns the maximum number of bytes of memory.Copy the code

5.4 GC Analysis mode

// --------------- jconsole use- Console Enter jconsole -1Select the local connection you want to debug and click Connect -2Select remote connection, enter user name, password connection// -------------- jvisualvm use- Go to the JDK installation directory and click Run jVisualvm. exe - select the running application directly on the right// --------------- jstat use (command line)Jstat <option> [-t] [-h] <pid> <interval> <count> Parameter description: Option can be set to -classAccording toClassLoadRelated information; -compilerAccording toJITInformation about compilation; -gcDisplay andgcRelated heap information; -gccapacityDisplay the capacity and usage of each generation; -gccauseDisplay information about garbage collection (pass -gcutil) while showing the triggers for the last or current garbage collection; -gcnewDisplay Cenozoic information; -gcnewcapacityDisplay new generation size and usage; -gcoldDisplay old and permanent generation information; -gcoldcapacityShows the size of the old age; -gcpermcapacityDisplays the size of the permanent generation; -gcutilDisplay garbage collection information. -printcompilationThe outputJITCompiled method information; -tYou can add to the printed columnTimestampColumn that displays how long the system has been running -hYou can specify how many rows you want to print a table header once for periodic dataintervalThe interval between each execution, in millisecondscountHow many times is used to specify the output records, will always default print case: | - :Jstat -cpmpiler pid| - viewpidFor 23814ClassLoadRelated information is printed once per second for a total of 5 prints:jstat -gc pid1000 | - displays information about the capacity of each generation:jstat -gccapacity pid| - according to the latestGCThe reason:jstat -gccause pid| - shows that the new generation of detailed information:jstat -gcnew pid: | - output new generation all the detailed information:jstat -gcnewcapacity pid| - according to old ageGCDetails of:jstat -gcold pid| - the output of the old s details:jstat -gcoldcapacitp pid| - look at each generation area using the percentage of the situation:jstat -gcutil pid

// ------------------- jmapusejmap [option] vmid
    
-dumpGenerated by:JavaHeap dump snapshot -heap: displayJavaHeap details -histo: shows the heap object statistics case: | - usejmapGenerate a snapshot file:jmap -dump:format=b,file=jsconsole.bin 7020| - to generate a snapshot of the normal operation of the jconsole instance: the JPS | - view the stack information: jmap heap pid | - using jmap generates snapshot files: jmap - dump: the format = b, the file = jsconsole. Bin7020    
    
// --------------------- Jhat usage
Step 1Jmap-dump :live,file=a.log pid Step2: Analyze the heap file jhat-j-xmx512m a1.log Step3: see HTTP://ip:7000/
Step 4Select <javascript expression to select> [from [instanceof] <class name> <identifier>]
    [where <javascript boolean expression to filter>]
    
(1)class nameisjavaThe fully qualified name of the class, such as:java.lang.String.java.util.ArrayList[CischarThe array,Ljava.io.Fileisjava.io.File[] (2) The fully qualified name of a class is not sufficient to uniquely identify a class, because differentClassLoaderThe same classes that were loaded injvmIs different type (3)instanceofIndicates that subclasses of a class are also queried, if not explicitlyinstanceof, the query is accurateclass nameClass specified (4)fromandwhereClauses are optional (5)javaDomain said:obj.field_name;javaArray representation:array[index] Example: (1) Query a string whose length is greater than 100select s from java.lang.String s where s.count> 100 (2) Queries arrays larger than 256 in lengthselect a from [I a where a.length> 256 (3) Displays a string that matches a regular expressionselect a.value.toString(a)from java.lang.String s where /java/ (s.value.toString(4) Display the file paths of all file objectsselect file.path.value.toString(a)from java.io.File file(5) Display allClassLoaderThe name of the classselect classof(cl).name from instanceof java.lang.ClassLoader cl(6) Query objects by referenceselect o from instanceof 0xd404d404 o
                   
https: / /www.cnblogs.com/baihuitestsoftware/articles/ 6406271.html 
                   
// ------------- jvm jinfo| - viewJVMParameters:jinfo -flags process_id| - viewjavaSystem parameters:jinfo -sysprops process_id                   
Copy the code

5.5 Expansion of the Pressure measurement Tool

// Common pressure measurement tool
1LoadRunner: Load testing tool that predicts system behavior and performance2Apache JMeter: Open source pressure testing product3NeoLoad: Load and performance testing tool4WebLOAD: Load testing tool from Radview, which can be used to test system performance and elasticity, as well as for correctness verification5Aliyun PTS: a SaaS performance testing platform with strong distributed pressure testing capability6Loadstorm: A cloud load test tool for Web applications that simulates mass clicks to test the performance of Web applications under heavy loads7CloudTest: An integrated stress test cloud platform that integrates performance and functional testing8Load Impact: A performance testing tool for DevOps that supports Web, Web, mobile, and API testing across a variety of platforms/ / use JMeter
    
Copy the code

5.6 use Jstack

> Step 1: to get the pid ps - ef | grep Java > Step2: View the resource process top-HP30275

printf "%x\n" 3440> Simply use jStack30275> View the printf process"%x\n" 17880  
jstack 17880|grep 45d8 -A 30
    
/ / check TimeWait


/ / Windows versions
netstat -ano |findstr "80" windows
netstat -an | find "TIME_WAIT" /C    
    
// Jstack counts the number of threads
jstack -l 28367 | grep 'java.lang.Thread.State' | wc -l    
    
    
// jstack Usage: Jstack [-l] <pid> (to connect to running process) Connect to active thread jstack -f [-m] [-L] <pid> (to connect to a hung process) Connect to blocked thread Executable > <core> (to connect to a core file) jstack [-m] [-l] [server_ID @]<remote server Options: -f to force a thread dump. Use when jstack <pid>does not respond (process is hung)
    -m  to print both java and native frames (mixed mode)
    -l  long listing. Prints additional information about locks
    -h or -help to print this help message
        
        
        
    
Copy the code

5.7 JOL use

JOL: <groupId>org.openjdk.jol</groupId> <artifactId>jol-core</artifactId> </artifactId> <version>put-the-version-here</version> </dependency> static Object generate() { Map<String, Object> map = new HashMap<>(); map.put("a", new Integer(1)); map.put("b", "b"); map.put("c", new Date()); for (int i = 0; i < 10; i++) { map.put(String.valueOf(i), String.valueOf(i)); } return map; } view object inside information: ClassLayout parseInstance (obj). ToPrintable object () to check the external information: including reference objects: GraphLayout. ParseInstance (obj). ToPrintable () to see objects take up the space is total size: GraphLayout. ParseInstance (obj). TotalSize ()Copy the code

5.8 the Thread Dump

Thread Dump is a very useful tool for diagnosing Java application problems. Each Java VIRTUAL machine has the ability to generate a thread-dump of all threads at a certain point in time. Although each Java virtual machine prints a slightly different thread dump, most of them provide a snapshot of the currently active thread and a stack trace of all Java threads in the JVM. The stack information typically contains the full class name and method executed, and if possible the number of lines of source code.1.Looking for memory leaks, it is common to load a large amount of data into the cache in the program;2.Deadlocked thread found;// Linux grab Dump method (20810 is jstack in Java directory)
jstack -l 20810 | tee -a /opt/jstack.log

// Simple learning:
// Vm information
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.251-b08 mixed mode):

// Thread info block:
Thread name - #36 - Thread type (daemon) - Priority (prio)
// tid: indicates the ID of the JVM thread
// nid: indicates the id of the system thread
// Thread status: in object.wait ().
// Start stack address: [0xae77d000]
"Attach Listener" #36 daemon prio=9 os_prio=0 tid=0x00007f5fec001000 nid=0x6658 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
	- None
        
// Stack informationState: WAITING (parking) Thread throws a node -at sun.misc.unsafe.park (Native Method) -parking to waitfor  <0x00000000e4e7bdf8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:107)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

/ / plan--> Dump --> dump --> dump --> dump --> dump --> dump --> dump --> dump --> dump --> dump --> dump Check if there are too many thread strikes in I/O, database, etc., locate the bottleneck cause, the request is not responding --> dump multiple times, and compare whether all runnable threads are performing the same method all the time// Common analysisDeadlocks, hot locksCopy the code

6. GC optimization

6.1 Prerequisite for GC optimization

> GC optimization is always the last task > Principle: > Minimize the number of objects transferred to the old age: Resize the new age space. > Reduce the execution time of Full GC: you need to set the old age space to an "appropriate" valueCopy the code

6. 2 GC optimization scheme

> Use StringBuilder or StringBuffer instead of String > Output log GC optimization parameters as little as possibleCopy the code

6. 3 Parameters to be considered in GC optimization

define parameter describe
Heap memory space -Xms Heap Area Size when Starting JVM Heap area size when starting the JVM.
-Xmx Maximum Heap Area Size Maximum heap memory size
Cenozoic space -XX:NewRatio Ratio of New area and Old area
-XX:NewSize New area size
-XX:SurvivorRatio Ratio of Eden area and Survivor area

6. 4GC Type This parameter is optional

classification parameter note
Serial GC -XX:+UseSerialGC
Parallel GC -XX:+UseParallelGC -XX:ParallelGCThreads=value
Parallel Compacting GC -XX:+UseParallelOldGC
CMS GC -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=value -XX:+UseCMSInitiatingOccupancyOnly
G1 -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC In JDK6, both parameters must be used together

6. 5 GC optimization process

1 > Monitor THE GC status. 2 > Analyze the monitoring result and determine whether GC is required. 3 > Adjust the GC type and allocate storage spaceCopy the code

6. 6 Memory Overflow And Analysis

> 1Stack overflow - Java. Lang. OutOfMemoryError:... java heap space..... -xmx and -xMS-too many visits and too much time per visit or too much data, Results in the release out - Java data. Lang. OutOfMemoryError: GC over head limit exceeded - system is in a state of high frequency of GC, and the effect of the recovery is still poor >2PermGen overflow - Java. Lang. OutOfMemoryError: PermGen space - the size of the system's constant pool is inflated by a large number of code, a large number of third-party packages, or a large number of constants in the code, or by injection of constants through the INTERN, or by dynamic code loading, etc.3The ByteBuffer allocateDirect () overflow - Java. Lang. OutOfMemoryError: Direct Buffer memory - if you use the allocateDirect method of ByteBuffer directly or indirectly, but not clear -XX:MaxDirectMemorySize >4Java. Lang. StackOverflowError - Java. Lang. StackOverflowError Xss is too small, we apply a lot of local call stack and content is stored in the users currently held by thread >5 java.lang.OutOfMemoryError: unable to create new nativeThread - indicates that an area of memory other than the heap cannot be allocated for the thread, either because the memory itself is insufficient or the heap is too large >6 java.lang.OutOfMemoryError: request {} byte for{}out of swap - This is usually caused by insufficient address spaceCopy the code

6. 7 Full GC Cause analysis and solution

// Cause: 1. The application created too many objects that could not be recycled quickly. 2. When the heap is split, even if there is a lot of free space, In the old generation may or may not directly at https://blog.gceasy.io/2020/06/02/simple-effective-g1-gc-tuning-tips/ https://www.oracle.com/technetwork/tutorials/tutorials-1876574.htmlCopy the code

Seven Tips

7. 1 Jar related

TODO
Copy the code

7 . 2 CPU

TODO
Copy the code

Eight other

What is the maximum heap size for 32-bit and 64-bit JVMS

In theory, a 32-bit JVM can have 2^32 heap memory. That is, 4GB of the maximum heap memory a 4-bit JVM can specify, which in theory can be 2^64Copy the code

# Direct memory (memory out of heap)

> NIO(New Input/Output) class, NIO(New Input/Output) class, A Channel and Buffer based I/O approach is introduced that can be usednativeThe library allocates memory directly out of the heap and then operates on a DirectByteBuffer object stored in the Java heap as a reference to that memory/ / contrastThe performance of direct memory IO reads and writes is higher than that of normal heap memory, especially when a certain amount of space is frequently requested. The difference is significant when multiple reads and writes are performedCopy the code

# Other tools

> GCEasy : 
Copy the code

# JMC Analysis process

// Step 1: Enable flight logging for the specified application

// Step 2: The analysis module. The flight record provides the following modules> general information: overview | | | | the JVM information system attributes record - CPU usage: can be judged whether the CPU to fill slowly - heap usage: Memory usage will cause garbage collection frequency, Redis usage (AOF/RDB persistence exception), -JVM information: Can learn the current type of virtual machine (recycling of different kinds of virtual opportunity to use different strategies, and using the JDK, configuration of the JVM parameters, etc.) > memory: overview | | garbage collection | GC time distribution of | | | GC configuration object statistics - an overview: - GC configuration: contains the GC configuration information and the corresponding collector type. - GC statistics time: - Garbage collection: contains the number of garbage collections and the elapsed time. -> Whether the frequency of garbage collection is normal or too frequent. -GC configuration: configure the number of GC threads, heap memory and other configuration details - Allocation: mainly TLAB allocation, > code: Overview | | | | hot method invocation tree exception error compile | | class loading - hot methods: whether code calls to the method of relevant reasonable (corresponding class will heap is too large, the object will not too much) - hot methods: Judge whether common objects will have multithreading risk and deadlock risk, whether the efficiency is too low - call tree: trace the root of the problem through the call tree - Exception error: judge whether there is an exception > thread: Hot overview | | | thread contention waiting time | | thread dump instance - | lock deadlock - judgement about the destruction of the thread - determine whether there is a thread switching losses (frequent switching), hot lock - judge that whether reasonable use tools such as the thread pool thread > IO: Overview | | | | socket file, speaking, reading and writing, speaking, reading and writing - this module can effectively analyze whether to read and write files caused by time delay or slow socket access to system > system information and environment variables, slightly > events: Proportion of events that occur, including Java Thread Park, Java Thread Start, Java Thread End, etcCopy the code

# SkyWalking Analyzing the flow

Skywalking is a link analysis tool, which is a good aid to quickly analyze bottlenecks// You need to click the refresh button to display the data
-javaagent:D:\java\plugins\shywalking\agent82\skywalking-agent.jar    
    
//- Dashboard: to see the status of each server - Topology: to analyze whether the server structure is reasonable (Redis server, mysql server, etc.) - Trace: To quickly determine slow SQL, slow interface - Performance analysis: (The created endpoint is the endpoint in the trace, and you can directly trace to the corresponding code line after clicking analysis)Copy the code

Optimize the process

9.1 Pressure testing process

> Select * from 'Mysql' where 'redis' =' Mysql 'where' redis' = 'Mysql' where 'redis' =' Mysql 'where' redis' = 'Mysql' where 'redis' =' Mysql 'where' redis' = 'Mysql' where 'redis' =' tomcat '=' TCP ' Random port, port total limit) - LDAP connections - OpenFile connections > Analyze GC -1To view the CPU usage, run the top - command2To view the usage of a specified process, run the top -hp [process ID] - command3Analyze the current process stack: jstack [process ID] > jstack_01 -4To check GC status: jstat -gcutil [process ID]1000
	- 5Check the heap memory: jmap -histo [process ID] > jmap.txt -6To print a snapshot of the heap memory: jmap-dump :format=b,file=aa.bin1232134> Optimize GC// GC is performed at 80
	- -XX:CMSInitiatingOccupancyFraction=80 -XX:+UseCMSInitiatingOccupancyOnly
	// Keep the GC logs--xx :+PrintGCDetails -xx :+PrintGCDateStamps -xx :+ printheapatgC-xloggc :gc.log > Load balancing - Determine whether the load mode is survey or pressure// Optimize the Cenozoic volume
-Xmn350M -> -Xmn800M
-XX:SurvivorRatio=4 -> -XX:SurvivorRatio=8
-Xms1000m ->-Xms1800m
    
/ / metaspace optimization
-Xmn350M -> -Xmn800M
-Xms1000M ->1800M
-XX:MetaspaceSize=200M
-XX:CMSInitiatingOccupancyFraction=75    	
Copy the code

Appendix:

Jstat Parameter name

The parameter name describe
gc Outputs the currently available and used space (Eden, Survivor, and so on) for each heap region, the total number of GC executions, and the time spent by GC operations accumulated.
gccapactiy Output minimum space limit (MS)/maximum space limit (mx) for each heap region, current size, and the number of times GC was performed on each region. (No output of currently used space and GC execution time).
gccause Output the information provided by -gcutil as well as the cause of the last GC and the cause of the current GC
gcnew Output GC performance data of Cenozoic space
gcnewcapacity Output statistics on the size of the Cenozoic space.
gcold Output GC performance data of the old age space.
gcoldcapacity Output statistics about the size of the old age space.
gcpermcapacity Output statistics about the size of the persistent tape space.
gcutil Output the usage of each heap region, as well as the total number of GC executions and events spent by GC operations.
column instructions Jstat parameters
S0C Output the size of a Survivor0 space. Unit of KB. -gc -gccapacity -gcnew -gcnewcapacity
S1C Output the size of a Survivor1 space. Unit of KB. -gc -gccapacity -gcnew -gcnewcapacity
S0U Output the size of used space in Survivor0. Unit of KB. -gc -gcnew
S1U Output the size of used space in Survivor1. Unit of KB. -gc -gcnew
EC Output the size of the Eden space. Unit of KB. -gc -gccapacity -gcnew -gcnewcapacity
EU Output the size of Eden’s used space. Unit of KB. -gc -gcnew
OC Output the size of the old age space. Unit of KB. -gc -gccapacity -gcold -gcoldcapacity
OU Output the size of the old space used. Unit of KB. -gc -gcold
PC Output the size of the persistent generation space. Unit of KB. -gc -gccapacity -gcold -gcoldcapacity -gcpermcapacity
PU Output the size of the space used by the persistent generation. Unit of KB. -gc -gcold
YGC Frequency of Cenozoic spatial GC time occurrence. -gc -gccapacity -gcnew -gcnewcapacity -gcold -gcoldcapacity -gcpermcapacity -gcutil -gccause
YGCT Time spent in the new generation GC processing. -gc -gcnew -gcutil -gccause
FGC Full Number of times GC has occurred. -gc -gccapacity -gcnew -gcnewcapacity -gcold -gcoldcapacity -gcpermcapacity -gcutil -gccause
FGCT Full Time spent in GC operation -gc -gcold -gcoldcapacity -gcpermcapacity -gcutil -gccause
GCT Total time spent in GC operations. -gc -gcold -gcoldcapacity -gcpermcapacity -gcutil -gccause
NGCMN Minimum spatial capacity of Cenozoic era, unit: KB. -gccapacity -gcnewcapacity
NGCMX Maximum space capacity of the Cenozoic generation, unit: KB. -gccapacity -gcnewcapacity
NGC Current space capacity of the New generation, unit: KB. -gccapacity -gcnewcapacity
OGCMN Minimum space capacity in the old age, in KB. -gccapacity -gcoldcapacity
OGCMX The maximum space capacity in the old era, in KB. -gccapacity -gcoldcapacity
OGC The current system of space capacity in the old era, in KB. -gccapacity -gcoldcapacity
PGCMN Minimum space capacity of persistent generation, in KB. -gccapacity -gcpermcapacity
PGCMX Maximum capacity of persistent space, in KB. -gccapacity -gcpermcapacity
PGC The current capacity of the persistent generation, in KB. -gccapacity -gcpermcapacity
PC Current space size of the persistent generation, in KB -gccapacity -gcpermcapacity
PU Currently used space of the persistent generation, in KB -gc -gcold
LGCC Cause of the last GC -gccause
GCC Cause of the current GC -gccause
TT Aging threshold. Number of times a Cenozoic void survived before being moved to an old age. -gcnew
MTT Maximum aging threshold. Number of times a Cenozoic void survived before being moved to an old age. -gcnew
DSS The size of the survivor’s area, in KB. -gcnew

Check common VM configurations

instructions The command
Enable GC Log (Java8) -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:{file-path}
Enable GC Log (Java9) -Xlog:gc*:file={file-path}

Thank you

// This note is a summary and analysis of the notes, a long period of time, many knowledge points have been difficult to trace back to the source, if you miss some friends here, please understand the Java Technology station, A series of death upon seeing quite cool CSDN http://cmsblogs.com/?p=5140 http://blog.csdn.net/linxdcn/article/details/72896616 taro source, Good source blog http://www.iocoder.cn/ the nuggets laoge CSDN https://blog.csdn.net/briblue https://juejin.im/post/5c31dca7e51d45524975d046 And all who contributed to the articleCopy the code