Store how do delay-queue consumers store messages after they pull them? Third-party storage must meet the following requirements: High performance: Write latency must be low. An important function of MQ is peak clipping and valley filling. When selecting temporary storage, write performance must be high, and relational databases such as Mysql usually do not meet the requirements. High reliability: delayed message writing, cannot be lost, need to persist, and backup low storage cost: can support a large number of message storage, (Redis storage cost is too high). Sorting: Messages can be sorted by a certain field. Delayed messages need to be sorted by time. Unlike ordinary messages, which are sent first and consumed first, delayed messages need to be sorted. For example, if a message is sent with a delay of 10 seconds and then a message is sent with a delay of 5 seconds, the later message is consumed first. Support for long time retention: Some services delay messages for several months or even longer, so delayed messages must be able to be retained for a long time. However, too long a delay is usually not recommended, the storage costs are high, and the business logic may have changed and no longer need to consume these messages. Based on the above conditions, RocksDB was chosen to store data: high performance embedded KV storage engine. Data is persisted to disk. Based on LMS storage, keys are naturally sorted and iterators are iterated by key order.Copy the code

What is RocksDB

Embedded Database, based on Google levelDB secondary development, greatly reduces traditional RPC data access

  1. This mode applies to multi-CPU scenarios
  2. Efficient use of Storge (higher IOPS, compression, less write wear)
  3. Not a distributed DB, but an efficient, high-performance, single-point database engine
  4. Store it in keys order, and a background Compaction wipes out duplicate and deleted keys.
  5. Pluggable architecture for easy replacement of components and tuning