Original: Monkey World (wechat official ID: Cxytiandi), welcome to share, please reserve the source.
On some business relationships between multiple teams, internal interactions can occur in an Rpc fashion. Some businesses do not really need strong association, so message queues are used for decoupling operations. For example, add points after placing an order, send SMS notification of this kind of operation.
When using message queues, one of the biggest concerns is will messages be lost?
Lost here means either you had a problem with the program and didn’t send it. Or it was sent, but at some point the message was lost, causing the consumer not to receive the message you sent, causing a business problem.
There are many solutions to message loss. Instead of talking about the technical aspects of preventing message loss, this article will discuss another topic: are messages sent at all?
This kind of problem is more likely to occur in the scenario of message decoupling between multiple teams. A student from another team comes to you and asks whether your message has not been sent. The state of the data on my side has not been transferred, so it is certain that the message has not been received.
Then you go to the console of the message queue and do a query for the message. The main problem is that this data is several months old, the sending record has not been kept for such a long time, so it is now in such a state of undisputable.
People say you didn’t send it, but you can’t prove that you did. So you’ll have to carry the pot yourself. This is the topic of today’s talk, everything should leave a voucher, easy to trace back, especially key business scenarios.
Stores send records to the database
Since you want to leave your credentials, you need to store them. The amount of messages in the message queue is large, and will not be stored permanently. Generally, it is stored in the last few days. Therefore, the message queue is directly used to check whether it is only suitable for the latest messages, and the longer time can not be traced.
After a message is sent, it can be directly stored in the database for future query. In fact, like SMS records, SMS also often encounter that I did not receive this message, is not sent ah and so on.
Storage needs to consider is the volume of the problem, if your daily message volume is large, but also need to store permanent data, then have to split, or use other external database for separate storage, such as MongoDB NoSql.
Store send records to log
Another solution is to simply send a log, because most companies have a unified log platform to collect and store logs. This solution does not occupy DB storage or a separate set of Nosql storage, relatively easy.
Note, however, that the log storage time can be set for some cloud vendors’ log services, because the larger the number of logs, the higher the cost.
In a long time ago encountered a message is actually printed log, but the storage time is only the latest month. When someone asks you if the message was sent, the log is gone, so we still need to store it permanently.
Permanent storage means an increase in cost. In fact, we can classify logs. Common logs can be stored for a shorter time, and some useful logs can be collected separately for output and permanent storage. This kind of log volume is relatively small, the cost is controllable.
Send directly from the local message table
In addition to using open source message queues, many companies also use local message tables, separate message services, and database-based message queues to send messages. These forms feature db-based persistence, so there is natural support for finding out if a message has been sent. Of course, it is also possible that even after a large amount of sub-database sub-table, or later expansion or regular archiving, as long as the data is still in the pot can not be carried back.
About the author: Yin Jihuan, a simple technology lover, the author of Spring Cloud Micro-service – Full stack Technology and Case Analysis, Spring Cloud Micro-service Introduction combat and Progress, the founder of the public account Monkey World.