This is the first day of my participation in Gwen Challenge

Summary:

We all know that redis is based on C language, and C string is judged to end with ‘\0’, is a special character storage is not safe. So how does Redis store binary security?

The highlight – strings in Redis

Redis is still useful for C strings, such as literals that do not need to be modified and log messages that reids output. The string redis that needs to be modified is abstracted into our SDS object: Simple Dynamic String (SDS). The source code for this section corresponds to the two files

sds.h

And SDS. C

The definition of SDS

Each SDS.h/SDSHDR structure represents an SDS value as follows:

Struct SDSHDR {// Record the number of bytes used in the buF array // is equal to the length of the string held by the SDS int len; // Record the number of unused bytes in the buF array. Char buf[]; char buf[]; };Copy the code

Our buf holds our true string contents terminated at \0 (changed to a pointer to different structures depending on length after version 3.2), and len is the length of our string contents. You can also see that there is a free subparagraph. The function of this subparagraph is kept in suspense and we say 😄

The role of free in SDS

How does Redis avoid the memory overflow problem that can cause strings to be overwritten when used incorrectly in C?

First we insert a key in redis:

> set demo hello;

The SDS objects at this time are:

struct sdshdr {
    int len; = 5
    int free; = 0
    char buf[]; = 'hello'

};
Copy the code

> set demo ‘hello world’;

The SDS objects at this time are:

struct sdshdr {
    int len; = 11
    int free; = 11
    char buf[]; = 'hello world'

};
> set demo 'hello world wang';
struct sdshdr {
    int len; = 16
    int free; = 6
    char buf[]; = 'hello world wang'

};
Copy the code

After we set hello world, Redis will determine whether we need to apply for memory from the operating system according to our free sub-segment. If free is sufficient, we will not apply for memory. In order to avoid the overhead of requesting memory from the operating system more than once, Redis will apply for twice as much memory as len (if len is less than 1M, it will only apply 1M more), which explains why our free is 11. This is the space preallocation of Redis.

We can look at the redis-5 source code, which is the strategy for allocating memory, which is the method for scaling XDM, we can look at SDS sdsMakeRoomFor(SDS S, size_t addlen)

Another concept of lazy space free involves our free subsegment. When we reduce the redis string content, Redis does not immediately free memory but simply modify the free subsegment for next use.

conclusion

  • Because SDS stores len, we get the redis string length as an O(1) operation.
  • Len avoids the memory overflow problem
  • Through the memory allocation strategy, the number of memory allocations for string modification is reduced