preface

MySQL > select * from ElasticSearch; select * from MongoDB; select * from ElasticSearch; select * from MongoDB; If the heterogeneous data synchronization between ElasticSearch and MongoDB is carried out by calling the API of other departments, it will definitely increase the communication cost and the probability of bugs. Therefore, we need another way to solve the asynchronous data synchronization of MySQL.

The solution

1. Alicanal is used for heterogeneous data synchronization

When MySQL executes an SQL statement, it records the executed SQL statement in binlog. Canal listens to the executed SQL statement and calls the Java code that has been written to add data to ElasticSearch or MongoDB to realize heterogeneous data synchronization of MySQL.

After canal is used, the problem of heterogeneous data synchronization can be easily solved, but the problem of decoupling between multiple departments cannot be well solved. Therefore, other ways are still needed to solve the problem of decoupling between multiple departments.

Message queue is used to synchronize heterogeneous data

We can add a message queue, between multiple database when MySQL data changes, send a message data changes to the message queue, subscribe to the message queue and other database after receiving the message data changes can modify their own data, both to ensure consistency of heterogeneous data, It also ensures the decoupling between multiple departments.

At the end

If you are concerned about the loss of data due to message queuing, you can perform a daily end task every day to synchronize data through scanning.

Thanks for watching, please give it a thumbs up if it’s helpful. Thank you.