“This is the 11th day of my participation in the First Challenge 2022. For details: First Challenge 2022

Abstract:

Many DBA students often encounter the problem of real-time synchronization from one database to another. Homogeneous data is relatively easy, but it is difficult to synchronize heterogeneous data, multiple tables and large amount of data. I personally tested a way, can be very convenient to complete the real-time synchronization of Oracle data to MongoDB, share with you, I hope it will be helpful to you.

Real-time synchronization of Oracle data to MongoDB and other non-relational databases should be the requirements of many students, now NOSQL database is more and more common. The data synchronization tool for this test is Tapdata Cloud, a good thing that is free forever.

Step 1: Configure the Oracle connection

  1. Click [Connection Management] in the left menu bar of Tapdata Cloud operation background, and then click [Create Connection] in the upper right corner of [Connection List] in the right area to open the connection type selection page, and then select Oracle

  2. On the displayed connection information page, enter the required configuration information.

[Connection name] : Indicates the name of the connection. The name of multiple connections must be unique

Database address: indicates the DATABASE IP address/Host

[Port] : database port

[Database name] : TapData Database connection takes a DB as a data source. Db here refers to the database in a database instance, not a schema.

[Account Number] : account that can access the database

Key code: password of the database account

[Time zone] : Time zone of the database is used by default. If the time zone is specified, the specified time zone is used

Step 2: Configure the MongoDB connection

  1. As in the first step, click [Connection Management] in the left menu bar, and then click [Create Connection] in the upper right corner of [Connection List] in the right area to open the connection type selection page, and then select MongoDB

  1. On the displayed connection information configuration page, enter the required configuration information one by one. After the configuration is complete, save the test connection.

Step 3: Select the synchronization mode – full/Incremental/Full + Increment

Enter the Tapdata Cloud operation background task management page and click the Add Task button to enter the task setting process

According to the connection just established, select the source and destination.

Select the libraries and tables to be synchronized based on data requirements. If you need to modify table names, you can set the table names of the target end in batches by using the table name batch modification function on the page.

After setting the above options, select the synchronization type. The platform provides full synchronization, incremental synchronization, and full + incremental synchronization, and set the write mode and read quantity.

If **** full and incremental synchronization is selected, **** automatically enters the incremental synchronization state after the full task is completed. In this state,The Tapdata Agent continuously listens for data changes at the source end(including: write, update, delete), and write these data changes to the target end in real time.

Click Task Monitoring to view the task execution details page.

 

Part four: data verification

After the completion of the general synchronization, I am used to the data check, to prevent stepping pits.

Tapdata CloudThere are three verification modes. I usually use the fastest fast count verification mode, which only needs to select the table to be verified without setting other complicated parameters and conditions. It is simple and convenient.

If it is not enough, you can select full-field verification of the table. In addition to selecting the table to be verified, you also need to set index fields for each table.Advanced verification is also supported for full-field verification of the table. Advanced check allows you to add JS check logic to check source and target data.

Another verification method is associated field value verification. When creating the associated field value verification, you need to select the table to be verified and set the index field for each table.

Perfect support for SQL->NOSQL, simply not too convenient. The above is the real-time synchronization of Oracle data to MongoDB operation share, hope the above operation share is helpful to you! Code word is not easy, reproduced please indicate the source ~

MySQL, SQL Server, PostgreSQL, DM DB, Kafka, etc. Full/incremental/full + increment, because the steps are the same, the other maps are not explained. DB2, Sybase, and Gbase databases are locked in grey state. They are under development. Synchronization of these databases will be supported in the future.

Clocked articles updated 33/100 days

You can like, collect, pay attention to, comment on me, there are database related questions to contact me or exchange yo ~!

​