In the distributed system, we know that the CAP theorem and the Base theory, the security and performance of data are negatively correlated. If the security of data is improved, the performance of data will decline. If the performance of data is improved, the security of data will decline. Let’s talk about this in terms of several middleware pieces.
mysql
When MySQL performance bottlenecks occur, we will do library and table separation, read/write separation, etc., read/write separation requires us to do master/slave replication, for read operations, we point to the slave library, for write operations, we point to the master library, let’s see the principle of master/slave replication.
When the business system commits the data, the master will save the changed data to the binlog file. Once written, the transaction commits successfully.
There is a thread in the slave that reads the binlog information from the master and then writes the relay file. Then he will have other threads that read the relay file and re-execute this information in the slave library. In other words, if an UPDATE statement is executed on the master, the exact same UPDATE statement is executed on the slave, so the data on both sides will be the same, and there will be a slight delay since the master executes first and then the slave executes later. If the statement is long, the delay will also be long, and of course the system stress will affect the delay time.
In addition to the delay, there is another big problem. When the binlog is written, the transaction is committed successfully. At this time, the master fails, so that the slave cannot read the binlog of this part, so there will be data loss, and the data between the two sides cannot be consistent. Therefore, we usually set the above form of asynchronous replication to semi-synchronous, i.e., semi-sync.
The difference between semi-synchronous replication and asynchronous replication is that when the master writes to the binlog, he initiates the synchronization of the data to the slave library. The slave writes to the relay file, and the ACK is returned to the master. At this time, the master will think that his transaction was committed.
In the case of multiple slaves, at least 1 ACK is returned before the transaction is committed. Although semi-synchronous replication solves the problem of data security, it needs to write the relay file from the library and return the ACK before committing the transaction. Compared with asynchronous replication, its performance is reduced.
monomer
The negative correlation of data security and performance also exists in the single unit. Here is a brief description of the MySQL update statement flow.
- When an update is executed, the data is first read from disk into the cache.
- Write to the undo file, which is used when our transaction is rolled back.
- Change the cache data, for example, change the name of id 1 from “Zhang San” to “Li 4”.
- Write to the redo cache, redo is mainly for the recovery of data in the event of a downtime restart.
- After writing to the Redo cache, write to disk.
- Write to the binlog file.
- After writing to binlog, commit to redo that you have written to binlog.
- Periodically write cached data to disk.
We update the data in the cache, so that we can guarantee the performance improvement. At the same time, for the security of data, undo, redo, binlog and other things are also introduced. We can look at the redo and binlog strategies for writing to disk.
In redo, we can choose 0, that is, do not write the cached data to the disk, so that we can quickly complete the redo operation. If the machine goes down at this time, the data will be lost before the data is written to the disk. Although the performance is improved, the data security is lost. If you select 1, the data will be safe, but the performance will be degraded because you have to write to disk to complete the redo operation.
Similarly, binlog writing to oscache improves performance, but server failure causes data from oscache to be unable to be written to disk in time, resulting in data security failure. If you write directly to disk, performance degrades.
redis
Similar to MySQL, Redis replication is also asynchronous replication. When the business system writes data to the master, it will synchronize the data to the slave through asynchronous replication. So similar to MySQL, when the business system thinks it has written data to Redis, the master will hang up at this time. But before the data is synchronized to the slave, his data is lost.
In another scenario, there is a split brain, in which Sentinel thinks the master has failed and elects the master again. At this time, the business system communicates with the master normally, and he submits the data to the original master node, but the data of the original master node cannot be synchronized to other nodes at this time. Resulting in inconsistencies in the data.
In this case, we will do the following configuration:
min-replicas-to-write 1
min-replicas-max-lag 10If at least one slave has not been synchronized for 10 seconds, then the master will pause to receive the request. So it’s not like the master is writing data all the time and the slave is not synchronized. If the above two scenarios occur, a maximum of 10 seconds of data will be lost. Although the data security is not strictly achieved, it also ensures that the inconsistency of the data will not exceed 10 seconds. After 10 seconds, the write performance drops to 0 because the data cannot be written.
RocketMQ
Let’s look at how broker synchronization is done based on DLEDGER.
First, when the master broker receives a message, it sets the message to unconmited and sends the message to the slave broker. When the slave broker receives the message, it sends an ACK to confirm it. If there are multiple slave brokers, When more than half of the slave brokers send ACK, the master broker will put the message in committed state. In this mechanism, the data security is guaranteed. When the master broker fails, we still have at least half of the slave brokers’ data intact, and the performance is also reduced due to the need for multiple slave brokers to ACK.
As a unit, asynchronous brush and synchronous brush are the same as MySQL’s redo writes to disk.
In addition, Kafka’s synchronization mechanism is similar to this, and it also has write oacache operations.
Zookeeper
ZooKeeper is based on CP model, so its data security can be guaranteed, but how about its performance? Let’s assume that the ZooKeeper cluster is unavailable when the master node fails and the master needs to be reselected. So how does ZooKeeper keep the data consistent?
So let’s say that the master synchronizes five slaves.
When the master returns directly without confirming whether or not it has been synchronized to the slave, the performance is highest, but the security is lowest, because the master data is not synchronized to the slave when the death, the data is lost.
When the master confirms that all slaves (5 in this case) have been synchronized before returning, performance is the lowest, but security is the highest, because no matter which slave his data is complete.
RocketMQ and ZooKeeper both choose a compromise between more than half of the slave synchronization, the success is considered. When we visit ZooKeeper, he will ask us to read the corresponding information according to the slave server which has been synchronized, so that the data we read is sure to be the latest.
