Redigo is a golang third-party library that operates on Redis, chosen because it is well documented and easy to use. A typical use of Redigo is as follows:
package main
import (
"github.com/gomodule/redigo/redis"
"log"
)
func main(a) {
conn, err := redis.Dial("tcp"."192.168.1.2 instead: 6379")
iferr ! =nil {
log.Fatalf("dial redis failed :%v\n", err)
}
result, err := redis.String(conn.Do("SET"."hello"."world"))
iferr ! =nil {
log.Fatalln(err)
}
log.Println(result)
}
Copy the codeOne thing to note here is that redis is only locally accessible by default and can be accessed remotely by changing bind in /etc/redis/redis.conf to the IP address of the machine where the service is located.
Although redigo is quite simple to use, its documentation points out one thing that needs our attention, which we can see in the original text in GoDoc:
Connections support one concurrent caller to the Receive method and one concurrent caller to the Send and Flush methods. No other concurrency is supported including concurrent calls to the Do and Close methods.
Translation:
A connection allows a single body to call Receive and a single body to call Send and Flush. Concurrent calls to Do and Close methods are not supported.
Out of programmer curiosity, I took a look at the source code for Redigo’s implementation of the Do method and figured out roughly why the Do function is concurrency insecure. Part of the source code is as follows:
func (c *conn) Do(cmd string, args ...interface{}) (interface{}, error) {
return c.DoWithTimeout(c.readTimeout, cmd, args...)
}
func (c *conn) DoWithTimeout(readTimeout time.Duration, cmd string, args ...interface{}) (interface{}, error) {
c.mu.Lock()
pending := c.pending
c.pending = 0
c.mu.Unlock()
if cmd == "" && pending == 0 {
return nil.nil
}
ifc.writeTimeout ! =0 {
c.conn.SetWriteDeadline(time.Now().Add(c.writeTimeout))
}
ifcmd ! ="" {
iferr := c.writeCommand(cmd, args); err ! =nil {
return nil, c.fatal(err)
}
}
iferr := c.bw.Flush(); err ! =nil {
return nil, c.fatal(err)
}
var deadline time.Time
ifreadTimeout ! =0 {
deadline = time.Now().Add(readTimeout)
}
c.conn.SetReadDeadline(deadline)
if cmd == "" {
reply := make([]interface{}, pending)
for i := range reply {
r, e := c.readReply()
ife ! =nil {
return nil, c.fatal(e)
}
reply[i] = r
}
return reply, nil
}
var err error
var reply interface{}
for i := 0; i <= pending; i++ {
var e error
ifreply, e = c.readReply(); e ! =nil {
return nil, c.fatal(e)
}
if e, ok := reply.(Error); ok && err == nil {
err = e
}
}
return reply, err
}
func (c *conn) writeCommand(cmd string, args []interface{}) error {
c.writeLen(The '*'.1+len(args))
iferr := c.writeString(cmd); err ! =nil {
return err
}
for _, arg := range args {
if err := c.writeArg(arg, true); err ! =nil {
return err
}
}
return nil
}
Copy the codeThe above three functions are implemented in the Conn. go file of redigo’s Redis package. In the DoWithTimeout method, we can see that it performs the sequential sending and receiving of data, and there is no lock in the function. Although the underlying implementation of TCP transmission in Golang is locking, which can ensure that data of one write operation will not be inserted by another write operation, there is still a faint smell of insecurity in the implementation of DoWithTimeout.
Let’s focus on the writeCommand method. From its implementation, we can learn that its role is mainly in for… Range sends the redis command to the redis-server for execution. At this point, we might notice that the function is unlocked if for… Range writes data to a global buffer, so concurrency is likely to result in data crossing. To verify this hypothesis, let’s move on to the writeArg implementation:
func (c *conn) writeArg(arg interface{}, argumentTypeOK bool) (err error) {
switch arg := arg.(type) {
case string:
return c.writeString(arg)
case []byte:
return c.writeBytes(arg)
case int:
return c.writeInt64(int64(arg))
case int64:
return c.writeInt64(arg)
case float64:
return c.writeFloat64(arg)
case bool:
if arg {
return c.writeString("1")}else {
return c.writeString("0")}case nil:
return c.writeString("")
case Argument:
if argumentTypeOK {
return c.writeArg(arg.RedisArg(), false)}// See comment in default clause below.
var buf bytes.Buffer
fmt.Fprint(&buf, arg)
return c.writeBytes(buf.Bytes())
default:
// This default clause is intended to handle builtin numeric types.
// The function should return an error for other types, but this is not
// done for compatibility with previous versions of the package.
var buf bytes.Buffer
fmt.Fprint(&buf, arg)
return c.writeBytes(buf.Bytes())
}
}
func (c *conn) writeString(s string) error {
c.writeLen('$'.len(s))
c.bw.WriteString(s)
_, err := c.bw.WriteString("\r\n")
return err
}
Copy the codeThe writeArg method calls different methods to write data based on the parameters passed in, but the underlying writeArg method calls the writeString method. In the writeString implementation, we see that Redigo writes all the data to BW. Bw is conn’s writter. That is, if the Do method is executed concurrently, all of these concurrent executables are writing to the same WRITter. This basically confirms my hypothesis.
After the DoWithTimeout function executes writeCommand, it calls the bw Flush method, which sends out all the data in the buffer.
// Flush writes any buffered data to the underlying io.Writer.
func (b *Writer) Flush(a) error {
ifb.err ! =nil {
return b.err
}
if b.n == 0 {
return nil
}
n, err := b.wr.Write(b.buf[0:b.n])
if n < b.n && err == nil {
err = io.ErrShortWrite
}
iferr ! =nil {
if n > 0 && n < b.n {
copy(b.buf[0:b.n-n], b.buf[n:b.n])
}
b.n -= n
b.err = err
return err
}
b.n = 0
return nil
}
Copy the codeFrom the code, we can see that after calling the b.w.rite method, there is an operation to determine whether the length of the data written is equal to the length of the data in the buffer. Redigo is not locked during the call to Do, so it is likely that other executants will write data to Writer’s buffer during the Flush process. A short write error occurs when the length of the written data is smaller than the buffer length after the b.w.rite call.
We can write a program to test this:
package main
import (
"github.com/gomodule/redigo/redis"
"log"
"sync"
)
func main(a) {
conn, err := redis.Dial("tcp"."192.168.1.2 instead: 6379")
iferr ! =nil {
log.Fatalf("dial redis failed :%v\n", err)
}
wg := sync.WaitGroup{}
wg.Add(2)
go func(a) {
defer wg.Done()
result, err := redis.String(conn.Do("SET"."hello"."world"))
iferr ! =nil {
log.Fatalln(err)
}
log.Println(result)
}()
go func(a) {
defer wg.Done()
result, err := redis.String(conn.Do("SET"."hello"."world"))
iferr ! =nil {
log.Fatalln(err)
}
log.Println(result)
}()
wg.Wait()
}
Copy the codeShort write error:
The authors of Redigo recommend that we use connection pooling to ensure security in concurrency, and the implementation of Redigo connection pooling will be read together next time.
Read the source code to see how the open source author implemented the open source work, but also open your eyes to some better programming techniques, this habit is a good one to stick with.