Go provides goroutines (function bodies) and channels (channels) for concurrent programming by programmers. In other languages it is difficult to build a massive system with thousands of threads, but in Go it is easy to create hundreds, millions, or even tens of thousands of goroutines to perform concurrent tasks. Go strongly advocates concurrency based on CSP (serial communication model) theory (See Chapter 8), but also provides a traditional locking mode for data sharing between multiple Goroutines (see Chapter 9).
Chapter 8 Goroutine & Passage
Go motto – Don’t communicate by sharing memory, communicate by sharing memory.
8.1 The Goroutine & Go keyword
If you have two function calls that are not dependent on each other, that means they can be made concurrently. Here, you can think of Go’s Goroutine as the mini version of “threads” (although in reality, goroutines and threads are very different in number, as described in Chapter 9).
When the program starts, there is a goroutine that runs the main function. This is called the main Goroutine. If you want to start a new Goroutine, you simply add the go keyword before a series of function calls. As with defer, the go keyword must be followed by a function call, but you can replace it with a func(){… }() is considered a block of statements to be executed.
In the following example, the main function computes the Fibonacci sequence inefficiently, and outputs some information in the other Goroutine to indicate that the system is still running (so the two work simultaneously).
func fibonacci(x int) int {
//return fib(x, 1, 1)
return slowFib(x)
}
// This method of calculation will pass the calculation results, and is tail recursive, very efficient.
func fib(n, left, right int) int {
switch {
case n == 0:
return left
default:
return fib(n- 1, right, left+right)
}
}
// This method does a lot of double counting, so it is very inefficient.
func slowFib(n int) int {
switch {
case n < 2:
return n
default:
return slowFib(n- 1) + slowFib(n2 -)}}func main(a) {
// Create a Thread and start().
// This goroutine runs independently of the main goroutine.
go func(a){
for {
fmt.Print("still running... \n")
time.Sleep(250 * time.Millisecond)
}
}()
// The main goroutine keeps counting
fmt.Print(fibonacci(46))}Copy the codeNote that all goroutines are forced to terminate once main completes and returns. So don’t forget to use time.sleep (…) when testing some code in this article. Or func{} suspends or idles the main goroutine.
8.2 channel
Channels are Bridges that connect goroutine to goroutine, and each channel is a conduit of a specific type, an element type called a channel. The key word for the channel is chan, and we create a chan element of type int using the make function.
As with the Map hash, the make function returns a chan reference itself. When it is copied or passed into a function, the caller and the called actually get the same reference. The zero value of a channel is nil, and channels are comparable: it’s just a matter of determining whether two channels point to the same reference.
8.2.1 Establishing bidirectional channels
A channel can be used for two purposes: to Send or Receive messages, which are collectively referred to as communication. Go introduced the <- symbol to represent communication graphically: ch < -x means sending the value x to channel CH. This expression can be understood as a call; <-ch indicates that a message is received from channel CH. The return value of this expression is the element type of the channel.
ch := make(chan int)
// In another Goroutine you can send messages of the specified type like this int chan.
go func(a){
// do something....
var x = 100
// sends the value of the variable x to ch.
ch <- x
}()
Int Chan receives messages of the specified type in another goroutine.
go func(a){
// Assigns the value of ch channel to y.
var y = <- ch
fmt.Print(y)
}()
// This goroutine has no messages to receive.
go func(a){
// The goroutine will be blocked after running to this point.
var y = <- ch
// This string is never printed on the console.
fmt.Printf("get a new message! %v",y)
}()
// Note that once the main goroutine exits, all tasks end, so try blocking it here.
fmt.Print(fibonacci(46))
Copy the codeNote that calling <-ch means consuming data (whether or not it is used), and each message can only be consumed once. In this example, the 100 sent by the first Goroutine (calling it a “producer” Goroutine) has been received by another “consumer” Goroutine. If another “consumer” Goroutine calls < -ch, it will block until another goroutine sends a new message to the CH.
By the way, you can just value from the channel and not process it, except that the message is discarded.
go func(a){
// Takes the value from the channel, but does not process it.
<- ch
// ...} ()Copy the codeSimilarly, if a “producer” Goroutine sends a message to a CH channel without a “consumer” processing, it will not send the message (this is important, see buffering channels below).
go func(a){
// This message is "unprocessed"
ch <- 100
// Causes the message to be blocked.
ch <- 101
fmt.Print("The program will not execute here.")
}()
Copy the codeCurrently, the created channel has no buffer space, and CH only supports “discovery fetch”. Therefore, when either party wants to send/receive messages, it needs to ensure that the old message has been fetched or the new message has arrived. If the condition is not met, it will be blocked. Therefore, the channel with no buffer space is also called synchronous channel.
Two Goroutine X, Y using a synchronous channel to send and receive messages usually means that there is a strict sequential execution of X, Y. For example, Y must wait for X to enter a variable before it can continue, or X must process a variable and pass it before Y. Alternatively, they are interacting serially with a shared variable protected by a “synchronous lock”.
If X, Y have no (or cannot describe) order of execution, then X and Y are concurrent. We also refer to each message in the communication as an event. Of course, communication can carry no other information and only use the communication action itself to represent the completion of a task.
8.2.2 pipeline
Goroutines can be connected by chan to form a coherent assembly line. This assembly line can also be called a Pipeline. Such as:
from := make(chan int)
to := make(chan int)
// Name this goroutine producer.
go func(a) {
for i := 0; i < 100; i++ {
from <- i
log.Printf("sent message {%d} to channel [from].", i)
}
}()
// Name this goroutine consumer.
// It inputs the value x2 received from into the to channel.
go func(a) {
for {
i := <-from * 2
to <- i
log.Printf("convert message to {%d} and send to channel [to]", i)
}
}()
// Name this goroutine printer.
// It outputs a message to the channel.
go func(a) {
for {
log.Printf("message from channel [to]:{%d}", <-to)
}
}()
// Block main goroutine
time.Sleep(500 * time.Second)
Copy the codeThere are two channels between the three goroutines: from and to (named after the consumer). The program as a whole works fine, but we’d like to make some improvements: for example, consumers should not “echo” messages to the FROM channel or “withdraw” messages to the TO channel.
8.2.3 Restricting unidirectional Channels by parameter list
Restricting the flow of channels is done on the function’s parameter list. There are differences in writing between “send only” and “take only” channels:
// chan < -int indicates that this is a send only channel, which corresponds to the send action in the two-way channel.
// <-chan int indicates that this is a receive only channel, which corresponds to the receive action in the two-way channel.
func pipeline(out chan <- int,in <-chan int){
// If a one-way channel is incorrectly used inside a function, it will be detected at compile time.
out <- 100
<- in
// Incorrect usage:
<- out
in <- 100
}
Copy the codeTwo-way channels (such as chan int) are implicitly converted to be compatible with “send only” (chan < -int) or “take only” (<-chan int) channel types. However, a one-way channel can no longer be switched to a two-way channel.
8.2.4 Buffer channel
The capacity of the channel can be set proactively through the make function. When the capacity is 0, the buffer channel degrades to a synchronous channel.
// Passing a second argument after the make function creates a buffer channel with length.
bufChannel := make(chan string.3)
Copy the codeThere is a difference between a synchronous channel and an unbuffered channel of length 1. As mentioned earlier, in the case of synchronous channels, when the producer realizes that no consumer can receive the message, the producer will not send the message and blocks immediately. For a non-buffered channel of length 1, the “producer” checks only when it is ready to send a second message: if there is no “consumer”, it waits and blocks.
When a producer sends a message to a channel, the length of the channel is incremented by one. Similarly, when a consumer gets a message from a channel, the length of the channel is reduced by one. When the length of the channel is smaller than the capacity, both parties can send and receive messages without blocking. Logically, you can think of a two-way channel as a queue for a FIFO.
Now, cap(bufChannel) is used to query the capacity of a buffered channel, and Len (bufChannel) is used to query the length of a buffered channel at a given moment. However, if a program is running continuously and processing messages very quickly, the length is usually variable — capturing the length once in most cases does not make much sense, and it is recommended to capture the length multiple times and use statistics to determine whether the capacity of the buffer channel is set properly.
The following code block demonstrates two goroutines that execute almost concurrently. Since the program runs “once,” the consumer returns after len senses that there are no new messages in the channel.
ch := make(chan int.3)
// producer
go func(a) {
ch <- 1
ch <- 2
ch <- 3} ()// consumer
go func(a) {
// Do not let it run before the first goroutine, otherwise it will exit directly
// So a small delay is set here.
time.Sleep(10 * time.Millisecond)
for {
if len(ch) == 0 {break}
fmt.Print(<-ch,"\n")
}
}()
time.Sleep(20 * time.Second)
Copy the code8.2.5 Beware of Goroutine leakage
The following code creates three goroutines to send requests to Baidu, Github, and Gitee at the same time, but this function will only return the response received first.
func cdn(a) *http.Response{
resp := make(chan *http.Response)
go func(a) {
get, err := http.Get("https://www.baidu.com")
if err == nil {
resp <- get
fmt.Printf("Baidu request complete")}} ()go func(a) {
get, err := http.Get("https://www.github.com")
if err == nil {
resp <- get
fmt.Printf("Github request completed.")}} ()go func(a) {
get, err := http.Get("https://www.gitee.com")
if err == nil {
resp <- get
fmt.Printf("Gitee request complete.")}} ()return <-resp
}
Copy the codeOn the surface, there seems to be no problem with calling this function. But the Bug is inherent: as soon as any of the requests return, it means that the CDN () function will immediately return and unstack. For the two slower Goroutines, no subsequent goroutines will receive their messages, so they will be blocked until the main program exits. By calling the CDN () function, you can observe that until the main program is finished, the console simply prints “XXX request completed “.
This phenomenon is called a Goroutine leak. The two leaking Goroutines are not collected by the garbage collector. For this example, there are two ways to avoid a Goroutine leak:
First, either party calls close(RESP) after sending a message first (see below) to prevent the other party from sending it again, relying on the native synchronization channel to ensure that only one message is sent successfully at a time. When the next two Goroutines tried to send a message, they crashed instead of blocking — the crash was expected, it was just a matter of recovering in a delayed call and letting them exit “as if nothing had happened.”
func cdn(a) *http.Response{
resp := make(chan *http.Response)
go func(a) {
defer func(a) {
recover()
fmt.Printf("Other Goroutines have received responses, Routine1 quits.")
}()
get, err := http.Get("https://www.baidu.com")
if err == nil {
resp <- get
fmt.Printf("Baidu request complete")
close(resp)
}
}()
go func(a) {
defer func(a) {
recover()
fmt.Printf("Other Goroutines have received responses, Routine2 quits.")
}()
get, err := http.Get("https://www.github.com")
if err == nil {
resp <- get
fmt.Printf("Github request completed.")
close(resp)
}
}()
go func(a) {
defer func(a) {
recover()
fmt.Printf("Other Goroutines have received responses, Routine3 quits.")
}()
get, err := http.Get("https://www.gitee.com")
if err == nil {
resp <- get
fmt.Printf("Gitee request complete.")
close(resp)
}
}()
return <-resp
}
Copy the codeThe second idea is to set the RESP to a buffer channel of capacity 3, and then ensure that all goroutines run. There is also the introduction of sync.waitgroup, which can be considered a “multi-Goroutine safe” semaphore. Add(1) and Done() are buried at the beginning and end of each child call to indicate that the semaphore is + 1 on entry into the Goroutine and -1 on exit.
For their parent call CDN (), if the semaphore is non-zero it means that some of the child calls did not complete. The parent call can then be blocked by calling Wait() until the subsequent goroutine completes and finally resets the semaphore to zero. In order not to delay the normal return of the CDN (), we might as well hand it over to another Goroutine.
func cdn2(a) *http.Response {
resp := make(chan *http.Response,3)
var mu sync.WaitGroup
go func(a) {
// Enter goroutine with semaphore +1, exit Goroutine with semaphore -1.
mu.Add(1)
defer mu.Done()
get, err := http.Get("https://www.baidu.com")
if err == nil {
resp <- get
fmt.Printf("Baidu request complete")}} ()go func(a) {
// Enter goroutine with semaphore +1, exit Goroutine with semaphore -1.
mu.Add(1)
defer mu.Done()
get, err := http.Get("https://www.github.com")
if err == nil {
resp <- get
fmt.Printf("Github request completed.")}} ()go func(a) {
// Enter goroutine with semaphore +1, exit Goroutine with semaphore -1.
mu.Add(1)
defer mu.Done()
get, err := http.Get("https://www.gitee.com")
if err == nil {
resp <- get
fmt.Printf("Gitee request complete.")}} ()defer func(a){
// Calling mu.wait () directly causes the function to Wait for all responses before returning,
// This is not necessary for the function's intended function.
go func(a){
mu.Wait()
fmt.Printf("The rest of the requests have been processed.")
close(resp)
}()
}()
return <-resp
}
Copy the code8.2.6 Selection of buffer channel and synchronization channel
How you choose these two types of channels, as well as the length of the buffer channel, can affect the performance of your program. In a nutshell, “buffer channels” are not always silver bullets.
Synchronous channels provide mandatory synchronization guarantees, which can be useful in situations where a “send” must be matched by a “receive”, such as blocking queues in HTTP/1.1, where the browser must wait for the previous request to receive a response before continuing to send the next request.
In “pipelined” pipes, where “send” and “receive” are usually separated, a buffered channel is more suitable. Assuming X is the upstream worker of Y, X simply stores the processed data into the buffer channel for Y to process, and X can continue processing the new data sent to it by its upstream worker — the same goes for Y.
Moreover, when there is a certain difference in the processing efficiency of X and Y, the buffer channel buys time for the slow party. For example, if X produces faster, the longer buffer channel prevents X from being “idle” due to blocking, and all nodes of the pipeline can remain “continuously working”.
But if the processing efficiency of X and Y is significantly different, then the buffer channel is either full (efficiency of X >> efficiency of Y) or empty (efficiency of Y >> efficiency of X) most of the time. At this point, the nodes degenerate to a state of synchronous blocking. At this time, the solution should be to create more workers to balance the efficiency difference at work, rather than blindly expanding the capacity of buffer channel (which only delays the time of blocking, but cannot solve the fundamental problem).
8.2.7 Channel Closure
More formally, the producer calls close(…) when it thinks there are no new messages to send. Actively close the sender of the channel — this means that the channel is marked closed but is still available (for busy consumers), but subsequent messages to the channel will cause outages.
ch := make(chan int.3)
// producer
go func(a) {
ch <- 1
ch <- 2
ch <- 3
// Close the channel.
close(ch)
}()
Copy the codeSimilar to hash map, consumers receive messages with two values: the first value is the message itself, and the second bool value stands for “messages pending on this channel that are still useful whether or not the channel is actually closed.” If it has a value of false, it indicates that “it makes no sense to continue reading messages from this channel,” accompanied by a zero value for the message type.
To the consumer: a closed channel is always readable, reading the rest of the channel’s messages without blocking, or the message’s zero value and a false. So if left unchecked, consumers will be trapped in an endless cycle.
// consumer
go func(a) {
// Do not let it run before the first goroutine, otherwise it will exit directly
// So a small delay is set here.
time.Sleep(10 * time.Millisecond)
for {
// Use the hasMore variable to determine whether there are any more messages to be processed
i,hasMore := <-ch
if! hasMore {break}
fmt.Print(i,"\n")}} ()Copy the codeOnly the producer (or “sender”) of the message should have the power to close the channel – in a two-way channel, if the consumer “willfully” closes the channel, the producer could easily crash while sending a new message, and such high-risk code would not be detected at compile time.
In one-way channels, this constraint is even stricter: if a consumer tries to close a “accept only” channel from upstream, the error will be detected at compile time.
8.3 Select multiplexing
Suppose a Job has multiple workers, and different workers use independent channels to communicate messages to the main Goroutine (or some other listener Goroutine).
Unfortunately, since the code is executed sequentially, no matter which worker the listener decides to listen on first, other workers will be blocked. In the following code block, worker2’s message processing is delayed because the Listener first blocks for messages sent by < -CH1 (with a 5 second delay), and the 5 seconds are wasted by the listener.
ch1 := make(chan int)
ch2 := make(chan int)
// worker1
go func(a){
time.Sleep(5 * time.Second)
ch1 <- 1} ()// worker2
go func(a) {
ch2 <- 1} ()//listener
go func(a){
// Block waiting to receive synchronization signal from worker1
<- ch1
fmt.Print("worker1 has done.\n")
// After receiving the message from CH1, wait to receive the synchronization signal from Worker2
<- ch2
fmt.Print("worker2 has done.\n")
}()
// Block the main process
time.Sleep(10 * time.Second)
Copy the codeIf listeners are expected to be “highly responsive”, the direct way to do this is to create a number of listeners, including workers. Obviously, this would create a large number of idle and blocked Goroutines, which would be a waste of resources.
One way to improve this is to create a sentinel selector that listens exclusively for channels in the form of polling, and then creates a new Goroutine when it hears new messages on a channel. Thus, a single blocking selector is needed to listen for messages on multiple channels at the same time, and will only block if there are no new messages on any of the channels.
This idea, called blocking multiplexing, has been used for thread management in major OS kernels. In Go, the implementation is to register multiple channels using a SELECT statement (which has a syntax similar to switch), and then wrap a for loop around it to implement continuous polling.
go func(a){
for {
// Only one statement is executed at a time.
select {
case <-ch1: go func(a) {println("worker1 has done.")} ()case <-ch2: go func(a) {println("worker2 has done.")}}}} () ()Copy the codeOccasionally, however, a SELECT statement will face some choice, such as new messages on more than one channel, which is more common when “listening” on more than one buffered channel. However, only one case will be executed in a SELECT call, and which one will be executed will be random.
8.4 close the Goroutine
Sometimes, we need to have multiple Goroutines actively stop their own tasks, such as when the client suddenly cancels in the middle of uploading a file. Child goroutines do not automatically end with parent goroutines (because these goroutines are parallel and independent. “Parent” is just a visual description of the code’s call hierarchy, unless the parent goroutine is the master Goroutine.
We must set up a mechanism for the parent goroutine to tell its child goroutine to stop and exit if it deems it unnecessary to continue. Now suppose you have two child Goroutine calls that each output odd/even numbers to the console in seconds. The requirement now is that when any key is pressed on the console, the two Goroutines exit, leaving the main function still running.
func main(a){
// goroutine1 keeps printing even numbers
go func(a) {
var i = 0
for {
fmt.Printf("[goroutine1]:%v\n",i)
i = i+2
time.Sleep(1 * time.Second)
}
}()
// goroutine2 continues to print odd numbers
go func(a) {
var i = 1
for {
fmt.Printf("[goroutine2]:%v\n",i)
i = i+2
time.Sleep(1 * time.Second)
}
}()
for {
fmt.Printf("[main]:still running... \n")
time.Sleep(1 * time.Second)
}
}
Copy the codeIt may be possible to create a buffer channel long enough that the main Goroutine sends “enough” semaphores to make all goroutines receiving semaphores stop working. So how do you determine the number of Goroutines currently running? Not sure. Especially for complex systems, if goroutines are reproducible — one goroutine creates (multiple) goroutines at run time, or some goroutine runs out early and returns, this can cause the number of goroutines to float.
To solve such problems, instead of having the parent goroutine actively notify the child Goroutine, the child Goroutine should actively detect a semaphore of the parent Goroutine. This is done using a synchronous channel;
func main(a) {
done := make(chan int)
var cancel = func(a) bool {
select {
// Recall that the consumer can always get a value from a closed channel.
case <-done:
return true
default:
return false}}// goroutine1 keeps printing even numbers
go func(a) {
var i = 0
for {
if cancel() {
return
}
fmt.Printf("[goroutine1]:%v\n", i)
i = i + 2
time.Sleep(1 * time.Second)
}
}()
// goroutine2 continues to print odd numbers
go func(a) {
var i = 1
for {
if cancel() {
return
}
fmt.Printf("[goroutine2]:%v\n", i)
i = i + 2
time.Sleep(1 * time.Second)
}
}()
// Turn on another goroutine to listen for input
go func(a) {
bufio.NewScanner(os.Stdin).Scan()
close(done)
}()
for {
fmt.Println("[main]:still running...")
time.Sleep(1 * time.Second)
}
}
Copy the codeThe main program closes the Done channel after receiving input. The result: Cancel () immediately receives the nil value of the message (refer to the channel closure section) and uses it as a semaphore to stop the Goroutine. All of the goroutine children of the main function are checked by calling the local cancel() function before each iteration, which is told true and returns exit one after another.
Chapter 9 Shared Variables
9.1 race
Suppose a Goroutine X has three sequential events inside it: x1, x2, and x3. The other Goroutine Y also has three sequential events inside it, y1, y2, and y3. Each event is guaranteed to run sequentially in its own Goroutine, but if X and Y run concurrently, it is uncertain which event comes first between xi and yj. Given that multiple Goroutines concurrently call the same function and that the function always returns the correct result, the function is said to be concurrency safe.
However, not all functions can guarantee this, and one reason is because of data races. Here is an example of how a data race can affect the results of a program (this example is often used in database tutorials for concurrent transactions) :
var balance int = 100
var deposit = func(amount int) {
balance = balance + amount
}
var show = func(a) int {return balance}
Copy the codeThere are now two people sharing the Balance account. They made deposits into the account almost at the same time:
// A
go func(a) {
deposit(200)
}()
// B
go func(a) {
deposit(100)
}()
time.Sleep(1 * time.Second)
fmt.Print(show())
Copy the codeIn a serialized operating environment, there is no dispute that the final account balance is 400. However, in the case of concurrent execution, the situation is different: for example, when A executes to balance + amount, B happens to update the balance through the deposit function, which means that THE balance that A now holds is an expired data. Updating the account balance with this expired data will cause B’s update to be overwritten (the bank made 100 yuan out of the account).
balance = 100
-------------------------------------
Aread = 100 |
| Bread = 100
| Bwrite = 200
Awrite = 300 |
--------------------------------------
balance = 300
Copy the codeThis phenomenon is a type of data race. It can only happen if two or more Goroutines share a variable and at least one of them writes to it. However, this example is so lightweight that it’s hard to see how the program responds to unexpected results.
Another example leads to an obvious phenomenon: a “Schrodinger” subscript access. Both goroutines assign a value to an X, which may be 10 or 100 (depending on which goroutine is executed). Once the length of x is 10, the main function will eventually get an error because the subscript is out of bounds.
var x []int
go func(a) {
x = make([]int.100)
}()
go func(a) {
x = make([]int.10)
}()
time.Sleep(100 *time.Millisecond)
fmt.Print(x[99])
Copy the codeThere are three ways to eliminate data races — first, if data is immutable, it must be concurrency safe.
var x []int
const (
LENGTH = 100
)
// Both Goroutines always create slices based on constants.
go func(a) {
x = make([]int,LENGTH)
}()
go func(a) {
x = make([]int,LENGTH)
}()
time.Sleep(100 *time.Millisecond)
fmt.Print(x[99])
Copy the codeBut in a business where the update operation is mandatory (as in the bank account example), this approach is not practical. The second approach is to avoid changing variables under multiple Goroutines.
Going back to the bank account example, other Goroutine threads can only submit modification requests, which are sent to a Balance proxy, Goroutine, which can only modify balance or echo back. In other words, balance itself will be invisible to A and B, who are restricted to interacting with agents in the form of synchronous channels. This corresponds to the motto of the Go language at the beginning.
request := make(chan int)
response := make(chan int)
var deposit = func(amount int) {
request <- amount
}
var show = func(a) int { return <-response }
// A
go func(a) {
deposit(200)
}()
// B
go func(a) {
deposit(100)
}()
// Proxy
go func(a){
// Read from MySQL
var balance = 100
for{
select {
case a := <-request: balance += a
case response <- balance:
}
}
}()
time.Sleep(1 * time.Second)
fmt.Print(show())
Copy the codeSome variables that cannot be restricted within an entire Goroutine can be channelled from upstream to downstream. Within a single node of the pipeline, any serial operations can be made on this variable, but once the variable has been sent, the node is not allowed to modify the variable. To put it more bluntly: variables are first passed to one node for serial use, and then passed to the next node for serial use. The way this variable is shared is called serial restriction.
syncCh1 := make(chan int)
syncCh2 := make(chan int)
syncCh3 := make(chan int)
go func(in <-chan int,out chan <- int){
i := <-in
out <- i + 3
}(syncCh1,syncCh2)
go func(in <-chan int,out chan <- int){
i := <-in
out <- i * 2
}(syncCh2,syncCh3)
syncCh1 <- 1
// Compute (x + 3) * 2 in a serial-constrained manner.
fmt.Print(<-syncCh3)
Copy the codeThe third is the most familiar mutual exclusion mechanism.
9.2 Sync.mutex
Even without any additional tools, we can build a mutex with a buffer channel of capacity 1. This channel represents a binary semaphore, with messages being locked and no messages being unlocked.
var (
mu = make(chan int.1)
balance = 100
)
var deposit = func(amount int) {
balance = balance + amount
}
var show = func(a) int{return balance}
// A
go func(a) {
/ / lock
mu <- 1
// Here is the critical code
deposit(100)
/ / lock
<-mu
}()
// B
go func(a) {
mu <- 1
// Here is the synchronization code
deposit(200)
<-mu
}()
Copy the codeMutex is so versatile that Go provides Sync.mutex directly. The Lock() method adds a Lock, and the Unlock method releases a Lock:
var (
mu sync.Mutex
balance = 100
)
var deposit = func(amount int) {
balance = balance + amount
}
var show = func(a) int{return balance}
// A
go func(a) {
mu.Lock()
// Here is the synchronization code
deposit(100)
mu.Unlock()
}()
// B
go func(a) {
mu.Lock()
// Here is the synchronization code
deposit(200)
mu.Unlock()
}()
time.Sleep(1 * time.Second)
fmt.Print(show())
Copy the codeAll blocks of code sandwiched between Lock() and Unlock() are called critical sections. After the other Goroutine is locked by Lock(), the other Goroutine blocks until Lock() is executed (the principle is that the Lock provided by Go is not reentrant, that is, cannot be re-locked on an already locked head) until it obtains the mutex first. Thus, for the above example, this ensures that only one person from A and B can call deposit() at A time.
In addition, within a Goroutine, lock and lock release operations must correspond, otherwise other Goroutines waiting to release locks will fall into a dead state. In this case, it is better to add the lock release action to the deferred call stack, so that the function will always be guaranteed to release the lock resource after exit (either normal exit or short-circuit exit due to an internal error).
mu.Lock()
defer mu.Unlock()
Copy the codeA mutex protected call is concurrency safe at the expense of a little performance.
9.3 Shared lock sync.rwmutex
Assuming that the bank finds that most customers are reading rather than writing to accounts, it might be useful to consider setting the reads of some transactions as shared locks (read/write locks) rather than mutex locks. This way, if two Goroutines compete for the same resource, but neither of them writes to the competing resource, they can share it. In other words, only read-read locks can coexist.
var (
mu sync.RWMutex
balance = 100
)
/ /... For write operations, Lock and Unlock methods are still called, so duplicate code is omitted here.
var show = func(a) int{
mu.RLock()
defer mu.RUnlock()
return balance
}
Copy the codeTherefore, critical sections protected by shared locks (read locks) should not have side effects including writing data. In addition, shared locks have a more complex internal mechanism than mutex, so sync.rwmutex is only better than mutex if Goroutines are competing for reads, otherwise it is less efficient than mutex.
9.4 Delaying the initialization of sync.once
Because of selective branches such as if or switch, the program does not guarantee that all variables will be used at run time. In particular, if some resource is expensive to initialize, it makes more sense to declare it as a “lazy” singleton.
var resp map[string]string
func loadResource(a) {
fmt.Printf("Resp loading begins... \n")
if resp == nil {
resp = map[string]string{
"banner": "/banner.jpg"."body": "/body.jpg"."background": "/background.jpg",}}}func main(a){
loadResource()
// The main goroutine runs serially, so we always get the correct result.
fmt.Printf("resp = [%v]", resp["body"])}Copy the codeThis code is not “Goroutine safe” (in other languages, the phenomenon is called “thread-safe”). Now, assuming that two Goroutines perform initialization and access, and that one goroutine accesses the RESP long before it is initialized, a null value is given, which may trigger repeated initialization actions:
// A Goroutine is being initialized.
go loadResource()
// make a goroutine trying to read.
// If it does not observe that the RESP is initialized, it executes the loadResource() function itself.
go func(a) {
s,ok := resp["body"]
if! ok { loadResource() fmt.Printf("resp = [%v]",s)
}
else {fmt.Printf("resp = [%v]",s)}
}()
Copy the codeFor example, in the code above, the loadResource function might be called multiple times, which would waste system resources. The console may print:
Resp loading begins... Resp loading begins...Copy the codeAn effective way to do this is to use the Mutex lock sync.mutex:
go func(a) {
loadMu.Lock()
loadResource()
loadMu.Unlock()
}()
go func(a) {
loadMu.Lock()
s,ok := resp["body"]
loadMu.Unlock()
if! ok { loadMu.Lock() loadResource() loadMu.Unlock() fmt.Printf("resp = [%v]",s)
} else {fmt.Printf("resp = [%v]",s)}
}()
Copy the codeTypically, the initialization of an expensive resource is done only once, followed by only calls to the resource. Instead of mutex, use the shared lock sync.rwmutex.
var loadMu sync.RWMutex
go func(a) {
loadMu.Lock()
loadResource()
loadMu.Unlock()
}()
go func(a) {
loadMu.RLock()
s,ok := resp["body"]
loadMu.RUnlock()
if! ok { loadMu.Lock() loadResource() loadMu.Unlock() fmt.Printf("resp = [%v]",s)
} else {fmt.Printf("resp = [%v]",s)}
}()
Copy the codeGo provides a more convenient tool to help achieve “Goroutine safe” lazy loading. The following code block migrates loadResource to a sync.once variable:
var resourceOnce sync.Once
// worker1
go func(a) {
resourceOnce.Do(loadResource)
}()
// worker2
go func(a) {
s,ok := resp["body"]
if! ok {// sync.once Ensures that loadResource is called only Once globally.
resourceOnce.Do(loadResource)
fmt.Printf("resp = [%v]",s)
} else {fmt.Printf("resp = [%v]",s)}
}()
Copy the codeIts function call Do(loadResource) always guarantees that the loadResource function will be called only once in the context. If any subsequent Goroutine “accidentally” touches this loading function, the Do will be used for air conditioning.
9.5 Race detector
The harsh reality is that no matter how carefully you manipulate multiple Goroutines, data races are inevitable in a concurrent programming environment. Go provides convenient tools to help programmers test and record data races that occur when programs are actually run. If you need to use it, just add a -race argument to the go command. In GoLand IDE, this can be set by going to Edit Configuration -> Go Tool Arguments.
This parameter commands the Go compiler to build a version with a race detector based on our original code (so the compilation time may be longer, but the cost is acceptable compared to manually analyzing data races). The race detector detects the event flow along with the running of the program and reports the conflict cases. The following is the report printed by the author of the case program before analyzing the program through the race detector on the machine:
==================
WARNING: DATA RACE
Write at 0x000000443688 by goroutine 7:
main.loadResource()
C:/Users/liJunhu/go/src/awesomeProject/main/runFirst.go:13 +0x204
sync.(*Once).doSlow()
C:/Go/src/sync/once.go:66 +0x10a
sync.(*Once).Do()
C:/Go/src/sync/once.go:57 +0x72
main.main.func1()
C:/Users/liJunhu/go/src/awesomeProject/main/runFirst.go:27 +0x4b
Previous read at 0x000000443688 by goroutine 8:
main.main.func2()
C:/Users/liJunhu/go/src/awesomeProject/main/runFirst.go:31 +0x45
Goroutine 7 (running) created at:
main.main()
C:/Users/liJunhu/go/src/awesomeProject/main/runFirst.go:26 +0x94
Goroutine 8 (running) created at:
main.main()
C:/Users/liJunhu/go/src/awesomeProject/main/runFirst.go:30 +0xb6
==================
Copy the codeThe general meaning of the report is: Before Worker1 (Goroutine 7) attempts to initialize the resource RESP (address 0x000000443688), Worker2 (Goroutine 8) is about to call resp[“body”] to try to get a value, This leads to a data race between the two routines.
Note, however, that it only records races that occurred during this run and does not guarantee that they will occur in the future.
9.6 Goroutine & Threads
First, there can be a huge difference in quantity and volume. User threads reserved by the operating system occupy a fixed amount of stack memory (typically 2 MB), which is used to hold local variables produced during function calls. In contrast, a Goroutine occupies an unfixed amount of stack memory (similar to that of a user thread) and is usually very small (perhaps 2KB) when the program is first run. As the program progresses, the memory footprint of a Goroutine can be increased/decreased as needed, and even a Goroutine can take up 1 GB of space (which is larger than a regular thread).
User threads are scheduled by the operating system kernel. Every few milliseconds, a hardware clock will send the CPU interrupt signal, by calling the CPU kernel function called scheduler (this will make the CPU from user mode in kernel mode) to implement a context switch: namely save on one thread running state, and return the current thread running state, the entire process requires a certain amount of time cost. Assuming that the program switches threads frequently, this cumulative time cost can be significant.
The Go language implements goroutine switching using its own scheduler, which is based on the G-P-M model. In simple terms, the scheduler maps m Goroutines to N operating system user threads using the M: n technique. It works in much the same way as CPU scheduling, but it only cares about how the Go program schedules the Goroutine, not how the user thread is manipulated. In the meantime, the logic that maps the Goroutine to the user’s thread is managed by the “P” — Processor, the logic Processor in the model, which acts as a “mediator.”
The Go scheduler also does not rely on the hardware clock for switching, but is triggered by the Go’s own architecture. Compared to thread switching, it avoids the CPU falling directly into kernel state, so switching goroutine is less costly than switching threads.
Here we feed a GOMAXPROCS environment variable that represents the n in “m: n”. Typically, this number is equal to the number of cpus in the machine. For example, on an 8-core machine, a Goroutine is assigned to one of the 8 user threads. If one of the goroutines in a user thread is blocked by calling time.Sleep or channel communication, the other active goroutines in that user thread can be moved to another available user thread to continue executing.
However, if the execution of a Goroutine involves system calls or relies on functions implemented by other languages, another user thread is allocated to execute it, but this is not counted in the number of GOMAXPROCS. The Go language encourages a simple programming style, so it deliberately leaves no discernable identifier for Goroutine: the result of a function should be determined only by its argument list, not “who runs it”.