Redis is associated with memcached, and vice versa. Redis supports simple key-value data types and stores list,set,zset,hash and other data structures. Redis supports data backup in master-slave mode. Redis supports persistence of data, retention of data in memory to disk, reloading of data for use upon restart, etc. This might seem better than memcached, but isn’t it? To be is to be, and let us compare them one by one according to a few differences.
Network IO model
Memcached is a multi-threaded, non-blocking IO reuse network model. It is divided into listening on the main thread and worker child threads. The listening thread listens on the network connection, receives the request, passes the connection description word pipe to the worker thread to read and write IO. The multithreaded model can be multi-core, but it introduces cache coherency and locking issues, such as memcached’s most common stats command, which requires all memcached operations to lock the global variable.
Redis uses single-thread IO reuse model and encapsulates a simple AeEvent event processing framework by itself, mainly realizing Epoll, Kqueue and SELECT. For single-thread IO operations, single-thread can maximize the speed advantage, but Redis also provides some simple computing functions. Such as sorting, aggregation, etc., for these operations, the single-threaded model will severely affect the overall throughput, the CPU calculation process, the entire IO scheduling is blocked.
Data support type
Memcached stores and accesses data in key-value form and maintains a large HashTable in memory, reducing the time complexity of querying data to O(1) and ensuring high-performance data access.
Redis, compared to memcached, only supports simple key-value data types. It also provides storage for list,set,zset,hash, and other data structures.
For memory-based database systems such as Redis and Memcached, the efficiency of memory management is a key factor affecting system performance. The malloc/free function in traditional C language is the most commonly used method to allocate and free memory. However, this method has major drawbacks. First, for developers, malloc and free mismatch can easily cause memory leaks. Secondly, frequent invocation will cause a large number of memory fragments cannot be reclaimed and reused, reducing the memory utilization. Finally, as a system call, its overhead is much higher than that of a normal function call. Therefore, in order to improve the efficiency of memory management, efficient memory management solutions do not directly use malloc/free calls. Redis and Memcached both use their own memory management mechanisms, but they are implemented in very different ways. Here’s how they are managed.
By default, Memcached uses Slab Allocation to manage memory. The main idea of Memcached is to divide the allocated memory into blocks of a specified length to store key-value data records of the corresponding length according to the preset size. This completely solves the memory fragmentation problem. Slab Allocation is only designed to store external data. That is, all key-value data will be stored in Slab Allocation. Other Memcached memory requests will be requested using the normal malloc/free method. Because the number and frequency of these requests will not affect the overall system performance, the principle of Slab Allocation is quite simple. As shown in the figure, it first applies for a large Chunk of memory from the operating system, divides it into chunks of various sizes, and divides the chunks of the same size into Slab classes. Chunk is the smallest unit used to store key-value data. The size of each Slab Class can be controlled by a Growth Factor at Memcached startup time. Assuming the Growth Factor is 1.25, if the first Chunk is 88 bytes, the second Chunk is 112 bytes, and so on.
When Memcached receives the data from the client, it selects the most appropriate Slab Class according to the size of the received data. Then, it queries the list of free chunks in the Slab Class stored by Memcached to find a Chunk that can be used to store data. When a database expires or is discarded, the Chunk occupied by the record can be reclaimed and added back to the free list. As you can see from the above process, Memcached’s memory management is efficient and does not cause memory fragmentation, but its biggest drawback is that it wastes space. Because each Chunk is allocated a specific length of memory space, variable-length data cannot fully utilize this space. As shown in the figure, when 100 bytes of data are cached in a 128-byte Chunk, the remaining 28 bytes are wasted.
Redis memory management mainly through the source code zmalloc.h and zmalloc.c two files to achieve. To facilitate memory management, Redis stores the size of a block of memory in the head of the block after allocating it. As shown in the figure, real_ptr is the pointer redis returns after calling malloc. Redis stores the size of the memory block in the header. Size occupies a known size of memory, which is of type size_t, and returns ret_ptr. When memory needs to be freed, reT_ptr is passed to the memory manager. With ret_ptr, the program can easily calculate the value of real_ptr and pass real_pTR to free memory.
Redis records all memory allocations by defining an array of length ZMALLOC_MAX_ALLOC_STAT. Each element of the array represents the number of memory blocks currently allocated by the program, and the size of the memory block is the subscript of that element. In the source code, this array is zmalloc_allocations. Zmalloc_allocations [16] represents the number of memory blocks of 16bytes that were allocated. Zmalloc. c has a static variable used_memory that records the total amount of memory currently allocated. So, in general, Redis uses mallc/ Free wrapped, which is much simpler than Memcached’s memory management approach.
In Redis, not all data is stored in memory at all times. This is one of the biggest differences from Memcached. When physical memory runs out, Redis can swap some long-unused values to disk. Redis only caches information about all keys. If Redis finds that the memory usage exceeds a certain threshold, the swap operation will be triggered. Redis calculates which key values need to be swapped according to swappability = age*log(size_in_memory). The values corresponding to these keys are persisted to disk and erased from memory. This feature allows Redis to hold data that exceeds the size of its machine’s own memory. Of course, the machine’s own memory must be able to hold all the keys, after all, these data will not swap. At the same time, when Redis swaps the data in the memory to the disk, the main thread that provides the service and the sub-thread that performs the swap operation will share the memory. Therefore, if the data needs to be updated, Redis will block the operation until the sub-thread completes the swap operation. When reading data from Redis, if the value of the read key is not in memory, Redis must load the corresponding data from swap and return it to the requester. There is the issue of an I/O thread pool. By default, Redis blocks until all swap files are loaded. This policy is suitable for batch operations when the number of clients is small. However, if you apply Redis to a large web application, this is obviously not sufficient for large concurrency. So Redis runs we set the size of the I/O thread pool to do concurrent operations on read requests that need to load the corresponding data from the swap file, reducing the blocking time.
Memcached uses pre-allocated memory pools to manage memory using slabs and chunks of different sizes. Items are stored in appropriate chunks according to their sizes. Memory pooling saves the cost of applying for or releasing memory and reduces the generation of memory fragments. And new data may be culled while there is still plenty of space in memory.
Redis use field application memory to store data, and rarely use the free – the list to optimizing the allocation of memory, will exist in a certain extent, memory fragments, Redis command parameters according to storage, the data with expiration time alone in together, and call them temporary data, is a temporary data will never be eliminated, Even if there is not enough physical memory, swap will not discard any non-temporary data (although it will try to discard some temporary data), so Redis is more suitable for storage than cache.
Data storage and persistence
Memcached does not support persistence of in-memory data. All data is stored in in-memory.
Redis supports persistence. Redis provides two different persistence methods to store data to hard disks. One is snapshotting, which can write all data existing at a certain time to hard disks. Another method is append-only file (AOF), which copies the write command to the hard disk when it is executed.
Data consistency issues
Memcached provides the CAS command to ensure the consistency of multiple concurrent access operations to the same data. This is not guaranteed by the fact that Redis does not provide cas commands, but Redis does provide transactional capabilities that guarantee the atomicity of a sequence of commands without any interruptions.
Different cluster management
Memcached is a full-memory data caching system. Redis supports data persistence, but full memory is the essence of its high performance. As a memory-based storage system, the size of the physical memory of the machine is the maximum amount of data that the system can hold. If the amount of data that needs to be processed exceeds the physical memory size of a single machine, you need to build distributed clusters to expand the storage capacity.
Memcached does not support distribution per se, so distributed storage of Memcached can only be implemented on the client side using distributed algorithms such as consistent hashing. The following diagram shows Memcached’s distributed storage implementation architecture. Before sending data to the Memcached cluster, the client uses a built-in distributed algorithm to calculate the destination node for the data. The data is then sent directly to the node for storage. However, when the client queries data, it also calculates the node where the data is queried and sends a query request to the node to obtain data.
Redis prefers to build distributed storage on the server side rather than on the client side. Distributed storage is already supported in the latest version of Redis. Redis Cluster is an advanced version of Redis that implements distributed and allows single point of failure. It has no central node and linear scalability. Distributed storage architecture of Redis Cluster, nodes communicate with each other through binary protocol, and nodes communicate with clients through ASCII protocol. In terms of data placement strategy, Redis Cluster divides the value field of the whole key into 4096 hash slots. Each node can store one or more hash slots, that is to say, the maximum number of nodes supported by the current Redis Cluster is 4096. Redis Cluster also uses a simple distributed algorithm: crc16(key) % HASH_SLOTS_NUMBER.
To ensure data availability in a single point of failure, the Redis Cluster introduces Master nodes and Slave nodes. In a Redis Cluster, each Master node has two Slave nodes for redundancy. In this way, the downtime of any two nodes in the entire cluster will not result in data unavailability. After the Master node exits, the cluster automatically selects a Slave node to become the new Master node.
Like dot dot attention dot like.
Welcome Java engineers who have worked for one to five years to join Java architecture development: 878249276, group provides free Java architecture learning materials (which have high availability, high concurrency, high performance and distributed, Jvm performance tuning, Spring source code, MyBatis, Netty, Redis, Kafka, Mysql, Zookeeper, Tomcat, Docker, Dubbo, multiple knowledge architecture data, such as the Nginx) reasonable use their every minute and second time to learn to improve yourself, don’t use “no time” to hide his ideas on the lazy! While young, hard to fight, to the future of their own account!