In this issue, we have extracted five tips about Go language for your reference according to the practical problems encountered by the front-line students in the development process: Golang performance optimization Go Ballast, Golang performance analysis benchmark+ Pprof, Golang single test technique piling, a lock-induced online service OOM, Go concurrent programming memory synchronization problem. I hope I can help you improve your skills
Go Ballast for 01Golang Performance Optimization
The most common method of Go GC optimization is to adjust the GC pace to adjust the trigger frequency of GC, mainly by setting GOGC and debug.setgcpercent (). Here’s a quick note on the downside of setting up the GOGC:
1. GOGC sets the ratio imprecisely, making it difficult to precisely control the garbage collection threshold we want to trigger;
2. The GOGC setting is too small to trigger GC frequently, resulting in ineffective CPU waste;
3. When the memory usage of the program itself is low, the memory usage of the program itself is low after each GC. If the GOGC is set at a GC pace that is twice the heap after the last GC, this threshold can easily be triggered, and it is very easy for the program to incur additional consumption due to GC triggering.
4. The GOGC setting is too large. If these interfaces suddenly receive a large wave of traffic, the failure to trigger GC for a long time may result in OOM.
As a result, the GOGC is not very friendly for some scenarios, but is there any way to accurately control memory so that it can accurately control GC at multiples of 10G?
And that’s where Go Ballast comes in. What is Go Ballast? It is simple to initialize a very large slice throughout the Go application lifecycle.
func main() {
ballast := make([]byte, 1010241024*1024) // 10G
// do something
runtime.KeepAlive(ballast)
}
Copy the codeThis code initializes ballast, runtime.KeepAlive, and ensures that GC can’t recycle ballast.
By using this feature, you can ensure that the GC is triggered only at one time of 10G, which gives you more precise control over the timing of the GOGC.
02Golang performance analysis benchmark+ Pprof
When writing Golang code, the performance of the code may be poor due to improper coding or the introduction of some time-consuming operations. At this time, we are faced with the problem of performance optimization. We need to quickly identify the “major performance consumers”, find the performance bottleneck, and quickly carry out targeted optimization.
Golang is an excellent language and a great tool for performance analysis.
Benchmark + Pprof from Golang can help us quickly analyze the performance of the code, and analyze the CPU and memory consumption of the code. Through the analysis of CPU consumption, quickly find out the CPU time-consuming function, targeted optimization, improve the performance of the program. Through the analysis of memory consumption, we can find out the large memory consumption objects in the code, and also can carry out the investigation of memory leakage.
Benchmark is a benchmark function provided by go Testing framework. It benchmarks the code to be analyzed and generates CPU profile and MEM profile data into a file. The two profiles are analyzed to locate performance problems.
Pprof is Golang’s built-in CPU and memory analyzer, which is included in the Go Tool. It can analyze CPU profile and MEM profile files produced by benchmark to locate performance bottlenecks.
Pprof can perform performance analysis interactively on the command line, but also provides visual graphics that can be analyzed in a browser. Before using visual analysis, you need to install Graphviz.
The data displayed on the Pprof visual analysis page is intuitive and thoughtful. For functions that consume a lot of CPU and memory, the labels will be darker and the corresponding graphics will be larger, allowing you to find them at a glance.
Examples of the benchmark test command used in this analysis:
go test -bench BenchmarkFuncA -run none -benchmem -cpuprofile cpuprofile.o -memprofile memprofile.o
Copy the codeExamples of pprof visual view commands used in the analysis:
go tool pprof -http=":8080" cpuprofile.o
Copy the codeAfter you run the following command, the browser automatically opens the analysis page or manually opens it:
http://localhost:8080/ui/.
03Golang single test skills of piling
3.1 introduction
When writing a single test, there are times when you need to make a given function or instance method return a particular value. It overwrites the executable at run time through assembly language to jump the implementation of the objective function or method to the pile implementation, similar to the principle of hot patches. Here’s a brief introduction to using monkey to drive piles in the Go language.
3.2 the use of
3.2.1 installation
go get bou.ke/monkey
Copy the code3.2.2 Functional piling
Rewrite the function you need to pile using monkey.Patch to return the conditional dependent parameters you need in a single test:
// func.go func GetCurrentTimeNotice() string { hour := time.Now().Hour() if hour >= 5 && hour < 9 { return "An hour in the morning is worth two in the morning. } else if hour >= 9 && hour < 22 {return "" } else {return "Late at night, go to bed early"}}Copy the codeWhen we need to control the return value of time.now (), we can pile as follows:
// func_test.go func TestGetCurrentTimeNotice(t *testing.T) { monkey.Patch(time.Now, func() time.Time { t, _ := time.Parse("2006-01-02 15:04:05", "2022-03-10 08:00:05") return t }) got := GetCurrentTimeNotice() if ! Strings.Contains(got, "a good deal in the morning ") {t. rorf("not expectd, got: %s", got)} t. lash ("got: %s", got)}Copy the code3.2.3 Example method piling
Example business code is as follows:
Func (u *User) GetAge() int {t, err := time.Parse("2006-01-02", u.Birthday) if err ! = nil {return -1} return int(time.now ().sub (t).hours ()/24.0)/365} // GetAgeNotice getuser (u *User) GetAgeNotice() string {age := u.getice () if age <= 0 {return fmt.sprintf ("%s is mysterious, we don't know ta yet." , u.name)} return FMT.Sprintf("%s is % D this year, ta is our friend." , u.Name, age) }Copy the codeWhen we need to control the return value of GetAge called in the GetAgeNotice method, we can pile as follows:
// method_test.go func TestUser_GetAgeNotice(t *testing.T) { var u = &User{ Name: "xx", Birthday: "1993-12-20"} / / for object method piling monkey. PatchInstanceMethod (reflect the TypeOf (u), "GetAge." Func (*User)int {return 18}) ret := u.getagenotice () // Internal calls to u.Getage return 18 if! Strings.Contains(ret, "friend "){t.fatal ()}}Copy the code3.3 Precautions
There are two things to note when using monkey:
1. It cannot pile the functions that have been optimized inline. Therefore, during the single test, the inline optimization of Go language needs to be closed as follows:
go test -run=TestMyFunc -v -gcflags=-l
Copy the code2. It is not thread-safe and cannot be used in concurrent single tests.
04 a lock triggered by the online service OOM
4.1 First take a look at the problem code sample
func service(){ var a int lock := sync.Mutex{} { ... Lock.Lock() if(a > 5){return} {... // Business logic} lock.UnLock()}Copy the code4.2 Analyzing the cause of the problem
RD introduced lock when writing the code because map is not thread safe. However, when a program returns, it does not unlock, so the lock cannot be released and continues to occupy memory. In goroutine, when a mutex lock is locked, without unlock, the coroutine will not end until the request times out and the context cancels. Therefore, be careful not to do IO operations in the lock and make sure that after the lock is locked, Unlock operation is available. At the same time, observe the usage of machine memory and CPU when going online, and pay attention to the number of Goroutine when using Go to write programs to avoid memory leakage caused by excessive creation.
4.3 Goroutine Monitoring perspective
4.4 How to stop loss quickly
First contact OP to cut the flow of the problem room, and then immediately roll back all the on-line orders at the problem point, stop the loss and then solve the problem.
4.5 Ways that can be improved
Program design stage: large flow interface, imperfect program design, case consideration is not comprehensive enough, machine performance is not taken into account.
Offline test: Perform pressure test on interfaces with heavy traffic. Interfaces with heavy traffic are prone to memory leaks.
Release phase: Pay attention to the machine performance data when the high-traffic interface goes online.
05Go concurrent programming memory synchronization issues
On modern computers, writes to memory are cached in the processor’s local cache before being flushed back to memory if necessary.
Under this premise, when a program is running on multiple processors and multiple Goroutines are running on different processors, it is possible that the processor cache is not flushed to memory in time and other Goroutines read an expired value.
As in the following example, although goroutine A and Goroutine B’s accesses to variables X and Y do not involve race problems, unexpected execution results are still possible:
var x, y int
// A
go func() {
x = 1
fmt.Print("y:", y, " ")
}()
// B
go func() {
y = 1
fmt.Print("x:", x, " ")
}(
Copy the codeThe possible execution results of the above code are:
y:0 x:1
x:0 y:1
x:1 y:1
y:1 x:1
x:0 y:0
y:0 x:0
Copy the codeThe last two cases occur because goroutines are serially consistent, but the order of events seen by multiple Goroutines is not necessarily the same without explicit synchronization using channels or mutex.
That is, although Goroutine A must be able to sense writes to X before reading Y, it may not be able to observe other Goroutine writes to Y, in which case it may output an expired value of Y.
In order to avoid the last two cases, it is necessary to use synchronization primitives to force the data in the processor cache to be flushed back to memory before reading variables, so that no goroutine can read an expired cache value from the processor:
Var x, y int var mu sync.rwmutex go func() {mu.rlock () // defer mu.runlock () x = 1 fmt.Print("y:", y, "")} () go func () {mu. RLock () / / synchronization primitives defer mu. RUnlock () y = 1 FMT. Print (" x," x, "")} ()Copy the codeCommon Go synchronization primitives:
sync.Mutex
sync.RWMutex
sync.WaitGroup
sync.Once
sync.Cond
Copy the code【 Technical Gas Station 】
Baidu Programmer development guide to avoid pit (3)
Baidu Programmer development Guide to Avoid pitfalls (Mobile part)
Baidu programmer development guide to avoid pit (front-end)
Baidu engineers teach you tips for improving r&d efficiency quickly
Baidu front-line engineers talk about the changing cloud yuan
【 Technology gas station 】 Reveal Baidu intelligent test scale landing
Brief discussion on the three stages of Baidu intelligent test