Defer is also a special keyword in Go. It is mainly used to ensure that all the functions following it are executed during the program execution, such as closing connections and clearing resources.
1. Structural overview
1.1. The defer
type _defer struct {
siz int32 // The size of the parameter
started bool // Whether the command is executed
sp uintptr // sp at time of defer
pc uintptr
fn *funcval
_panic *_panic // Panic in defer
link *_defer // Defer is cascaded through the link attribute in the function execution flow
}
Copy the code1.2. panic
type _panic struct {
argp unsafe.Pointer // pointer to arguments of deferred call run during panic; cannot move - known to liblink
arg interface{} // argument to panic
link *_panic // link to earlier panic
recovered bool // whether this panic is over
aborted bool // the panic was aborted
}
Copy the code1.3 g
Since the defer Panic is all tied to the running G, here are some of the properties in G that relate to defer Panic
type g struct {
_panic *_panic // Panic list
_defer *_defer // defer consists of the advanced and out linked list, the same stack
}
Copy the code2. Source code analysis
2.1. The main
To begin with, let’s use go Tool to analyze the underlying functions
func main(a) {
defer func(a) {
recover() ()}panic("error")}Copy the codego build -gcflags=all=”-N -l” main.go
go tool objdump -s “main.main” main
▶ go tool objdump -s "main\.main"main | grep CALL main.go:4 0x4548d0 e81b00fdff CALL runtime.deferproc(SB) main.go:7 0x4548f2 e8b90cfdff CALL runtime.gopanic(SB) main.go:4 0x4548fa e88108fdff CALL runtime.deferreturn(SB) main.go:3 0x454909 e85282ffff CALL runtime.morestack_noctxt(SB) main.go:5 0x4549a6 e8d511fdff CALL runtime.gorecover(SB) main.go:4 0x4549b5 e8a681ffff CALL runtime.morestack_noctxt(SB)Copy the codeAs you can see from the decompile results, the defer keyword first calls Runtime.deferProc to define a deferred object, and then, before the function ends, calls Runtime.deferReturn to complete the function that deferred defined
The panic function calls Runtime.gopanic to implement the logic
To recover, runtime.gorecover is called
2.2. deferproc
Create a function that deferred execution based on the function fn defined after the defer keyword and the size of the argument, and hang this function on the linked list of _defer in current G
func deferproc(siz int32, fn *funcval) { // arguments of fn follow fn
sp := getcallersp()
argp := uintptr(unsafe.Pointer(&fn)) + unsafe.Sizeof(fn)
callerpc := getcallerpc()
// Get a _defer object and put it in the head of the G._defer list
d := newdefer(siz)
// Set fn PC sp for defer, etc., called later
d.fn = fn
d.pc = callerpc
d.sp = sp
switch siz {
case 0:
// Do nothing.
case sys.PtrSize:
The memory following _defer stores the address information for argp* (*uintptr)(deferArgs(d)) = *(*uintptr)(unsafe.Pointer(argp))
default:
// If it is not a pointer parameter, copy it into the memory space after _defer
memmove(deferArgs(d), unsafe.Pointer(argp), uintptr(siz))
}
return0()
}
Copy the codeThe seemingly simple function acquires a _defer object with newproc, adds it to the head of the current G’s _defer linked list, and copies the parameters or Pointers to the parameters into the memory space immediately after the _defer object
2.2.1. newdefer
Newdefer takes an *_defer* object and pushes it to the head of the G._defer linked list
func newdefer(siz int32)* _defer {
var d *_defer
// Determine how many sizeclass should be allocated according to the size of the deferclass. This is similar to how many sizeclass should be allocated according to the size of memory
sc := deferclass(uintptr(siz))
gp := getg()
// If sizeclass is within the specified sizeclass range, go to p of g binding
if sc < uintptr(len(p{}.deferpool)) {
pp := gp.m.p.ptr()
if len(pp.deferpool[sc]) == 0&& sched.deferpool[sc] ! =nil {
// The current size of caches ==0 and not nil, get a batch of caches from sched
systemstack(func(a) {
lock(&sched.deferlock)
for len(pp.deferpool[sc]) < cap(pp.deferpool[sc])/2&& sched.deferpool[sc] ! =nil {
d := sched.deferpool[sc]
sched.deferpool[sc] = d.link
d.link = nil
pp.deferpool[sc] = append(pp.deferpool[sc], d)
}
unlock(&sched.deferlock)
})
}
// If the cache to be sizeclass is not empty after being fetched from Sched, allocate it
if n := len(pp.deferpool[sc]); n > 0 {
d = pp.deferpool[sc][n- 1]
pp.deferpool[sc][n- 1] = nil
pp.deferpool[sc] = pp.deferpool[sc][:n- 1]}}// Neither P nor Sched found or had no corresponding sizeclass
if d == nil {
// Allocate new defer+args.
systemstack(func(a) {
total := roundupsize(totaldefersize(uintptr(siz)))
d = (*_defer)(mallocgc(total, deferType, true))
})
}
d.siz = siz
// Insert into the header of g._defer
d.link = gp._defer
gp._defer = d
return d
}
Copy the codeGet sizeclass according to size and cache sizeclass. This is the idea when allocating memory
The idea of a second level cache is really all over the go source code
2.3. deferreturn
func deferreturn(arg0 uintptr) {
gp := getg()
// Get the first and last declared g defer defer from the linked list
d := gp._defer
// Without defer, there is no need to do anything
if d == nil {
return
}
sp := getcallersp()
// If the sp of defer does not match the callersp of defer, it is possible that the deferred function of another stack frame has been called
ifd.sp ! = sp {return
}
// According to d. iz, retrieve the previously stored parameter information and store it in arg0
switch d.siz {
case 0:
// Do nothing.
case sys.PtrSize:
*(*uintptr)(unsafe.Pointer(&arg0)) = *(*uintptr)(deferArgs(d))
default:
memmove(unsafe.Pointer(&arg0), deferArgs(d), uintptr(d.siz))
}
fn := d.fn
d.fn = nil
// defer has been used and released,
gp._defer = d.link
freedefer(d)
// Jump to execute defer
jmpdefer(fn, uintptr(unsafe.Pointer(&arg0)))
}
Copy the code2.3.1. Freedefer
The functions used to release defer should have the same idea as the scheduler and memory allocation
func freedefer(d *_defer) {
// Determine the sizeclass of defer
sc := deferclass(uintptr(d.siz))
// If the size is beyond the specified size, it is directly allocated memory
if sc >= uintptr(len(p{}.deferpool)) {
return
}
pp := getg().m.p.ptr()
// the buffer corresponding to the local sizeclass is full, and the batch is transferred half to the global sched
if len(pp.deferpool[sc]) == cap(pp.deferpool[sc]) {
// Use g0 to transfer
systemstack(func(a) {
var first, last *_defer
for len(pp.deferpool[sc]) > cap(pp.deferpool[sc])/2 {
n := len(pp.deferpool[sc])
d := pp.deferpool[sc][n- 1]
pp.deferpool[sc][n- 1] = nil
pp.deferpool[sc] = pp.deferpool[sc][:n- 1]
// Start by strating the batch of defer objects that need to be moved into a linked list
if first == nil {
first = d
} else {
last.link = d
}
last = d
}
lock(&sched.deferlock)
// Put the list in the header of sched.deferpool with sizeclass
last.link = sched.deferpool[sc]
sched.deferpool[sc] = first
unlock(&sched.deferlock)
})
}
// Clear the properties of the current defer release
d.siz = 0
d.started = false
d.sp = 0
d.pc = 0
d.link = nil
pp.deferpool[sc] = append(pp.deferpool[sc], d)
}
Copy the codeThe idea of second-level cache, after in-depth understanding of the implementation and Scheduler analysis of Go-Goroutine, in-depth understanding of the principle of Go-channel and SELECT, in-depth understanding of go-garbage collection mechanism has been analyzed, not too much analysis
2.4. gopanic
func gopanic(e interface{}) {
gp := getg()
var p _panic
p.arg = e
p.link = gp._panic
gp._panic = (*_panic)(noescape(unsafe.Pointer(&p)))
atomic.Xadd(&runningPanicDefers, 1)
// Execute the G._defer linked list defer objects in turn
for {
d := gp._defer
if d == nil {
break
}
// If defer was started by earlier panic or Goexit (and, since we're back here, that triggered a new panic),
// take defer off list. The earlier panic or Goexit will not continue running.
// Normally, defer is removed after it has been executed, but since this one has not been removed, there are only two reasons: 1. There was panic 2 in this defer. Runtime. Goexit was triggered in this defer, but this defer has been executed and needs to be removed. If the first reason that caused this defer was not removed, then the panic also needs to be removed. Since panic has already been executed, add flag bits to Panic for subsequent removal
if d.started {
ifd._panic ! =nil {
d._panic.aborted = true
}
d._panic = nil
d.fn = nil
gp._defer = d.link
freedefer(d)
continue
}
d.started = true
// Record the panic that is running the defer.
// If there is a new panic during the deferred call, that panic
// will find d in the list and will mark d._panic (this panic) aborted.
// Bind the current panic to this defer. There may be a panic in the defer, in which case it will enter the d.started logic above and terminate the current panic since it has already been executed
d._panic = (*_panic)(noescape(unsafe.Pointer(&p)))
/ / execution defer. Fn
p.argp = unsafe.Pointer(getargp(0))
reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz))
p.argp = nil
// reflectcall did not panic. Remove d.
ifgp._defer ! = d { throw("bad defer entry in panic")}// Resolve the bind between defer and panic as the defer function has already been executed and would not have been executed if there had been panic or Goexit
d._panic = nil
d.fn = nil
gp._defer = d.link
// trigger shrinkage to test stack copy. See stack_test.go:TestStackPanic
//GC()
pc := d.pc
sp := unsafe.Pointer(d.sp) // must be pointer so it gets adjusted during stack copy
freedefer(d)
// Panic is recovered, there is no need to continue panic, continue to execute the rest of the code
if p.recovered {
atomic.Xadd(&runningPanicDefers, - 1)
gp._panic = p.link
// Aborted panics are marked but remain on the g.panic list.
// Remove them from the list.
// Remove aborted panic from the panic list, explained below
forgp._panic ! =nil && gp._panic.aborted {
gp._panic = gp._panic.link
}
if gp._panic == nil { // must be done with signal
gp.sig = 0
}
// Pass information about recovering frame to recovery.
gp.sigcode0 = uintptr(sp)
gp.sigcode1 = pc
// Call recovery to restore the current g scheduling execution
mcall(recovery)
throw("recovery failed") // mcall should not return}}// Prints panic information
preprintpanics(gp._panic)
// panic
fatalpanic(gp._panic) // should not return* (*int) (nil) = 0 // not reached
}
Copy the codeGp._panic. Aborted is explained here as an example
func main(a) {
defer func(a) { // defer1
recover()
}()
panic1()
}
func panic1(a) {
defer func(a) { // defer2
panic("error1") // panic2} ()panic("error") // panic1
}
Copy the code-
When panic(“error”) is executed
G._defer: G._defer->defer2->defer1
G. _panic list: G. _panic->panic1
-
When panic(“error1”) is executed
G._defer: G._defer->defer2->defer1
G. _panic list: G. _panic-> panIC2 ->panic1
-
Continue inside the defer1 function to recover()
The panic caused by Panic2 will be recovered, panic2. Panic2. Recovered = true. Remove panic that has already been executed
forgp._panic ! =nil && gp._panic.aborted { gp._panic = gp._panic.link } Copy the code
The logic of Panic can be sorted out:
When the program meets panic, it does not continue. It first mounts the current panic to the G. _defer linked list of the current G, and then executes the functions defined by the _defer object. If panic occurs again during the call of the function defer, The gopanic function is executed, and finally, the loop prints all panic information and exits the current g. However, if recover is encountered during the call to defer, scheduling continues (McAll (Recovery)).
Against 2.4.1. Recovery
Restore a PANIC g, re-enter and continue scheduling
func recovery(gp *g) {
// Info about defer passed in G struct.
sp := gp.sigcode0
pc := gp.sigcode1
// Make the deferproc for this d return again,
// this time returning 1. The calling function will
// jump to the standard return epilogue.
// Record the sp PC returned by defer
gp.sched.sp = sp
gp.sched.pc = pc
gp.sched.lr = 0
gp.sched.ret = 1
// Resume the execution schedule
gogo(&gp.sched)
}
Copy the code2.5. gorecover
Gorecovery simply set the flag bit g._panic. Recovered
func gorecover(argp uintptr) interface{} {
gp := getg()
p := gp._panic
// Depending on the address of argp, you need to determine whether it is called in the defer function
ifp ! =nil && !p.recovered && argp == uintptr(p.argp) {
// Set the flag bit, which will be judged by gopanic
p.recovered = true
return p.arg
}
return nil
}
Copy the code2.6. goexit
We also overlooked the fact that when we manually exit with runtime.goexit (), the defer function also executes, so let’s examine the situation
func Goexit(a) {
// Run all deferred functions for the current goroutine.
// This code is similar to gopanic, see that implementation
// for detailed comments.
gp := getg()
// Walk through the list of defer
for {
d := gp._defer
if d == nil {
break
}
// If defer has already executed, the panic bound to defer terminates
if d.started {
ifd._panic ! =nil {
d._panic.aborted = true
d._panic = nil
}
d.fn = nil
// Remove from the deferred list
gp._defer = d.link
/ / release the defer
freedefer(d)
continue
}
// Call the defer inner function
d.started = true
reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz))
ifgp._defer ! = d { throw("bad defer entry in Goexit")
}
d._panic = nil
d.fn = nil
gp._defer = d.link
freedefer(d)
// Note: we ignore recovers here because Goexit isn't a panic
}
// Call goexit0 to clear the attributes of the current g and resume scheduling
goexit1()
}
Copy the code2.7. Graphical interpretation
The source code is not very difficult to read, if there is any doubt, I hope the following GIF can solve your doubts
I’m sorry for the poor drawing
Step analysis:
- L3: Generate a DEFER1 and put it on the g._defer linked list
- L11: Generate a DEFER2 and mount it to the G._defer linked list
- L14: Panic1 calls gopanic to place the current panic on the G. _PANIC linked list
- L14: Because panic1, extract from the g._defer linked list header to DEFER2 and start execution
- L12: Perform defer2, another PANIC, and mount it to the G. _PANIC list
- Defer2 has already been removed from the list, and defer2 was triggered by Panic1. Skip defer2 and abort panic1
- L12: Continue to extract the next in the G._defer linked list, to DEFER1
- L5: Defer1 Perform recover to recover panic2, remove the list, and determine the next panic (panic1). Panic1 is aborted by defer2, remove Panic1
- Defer1 completes execution, remove Defer1
3. Associate documents
- Level 2 Cache, Sizeclass: In-depth understanding of the Go-garbage collection mechanism
- Gogo GoEXit0 scheduling: In-depth understanding of go-Goroutine implementation and Scheduler analysis
4. Reference documents
- “Go Language Learning Notes” — Rain Stains