Refer to the two-stage transaction (2PC) idea of multi-threaded transactions

preface

Put forward the demand first, do not fit the technical implementation of the business is playing rogue. I have a 50W ~ 100W large data volume excel import here. Import the data of the previous month in the current month. Real time requirement is not high. Import sub-order details. Import all services or none if an error occurs. The old code implementation reads line by line, then reaches the number of execution lines, for batch inserts into the database, execution time is about 10 minutes. Up to 1 minute after optimization. I suggest that we change space for time and make an optimization, because only a few business people are using this function. It is generally used after the pilot into the data, so it is safe to import boldly.

Thought analysis

Post the simple idea of the code, with the idea of the other is to copy the gourd gourd gourd gourd gourd gourd.

public class TestDemo {
    // Make sure that the weak synchronization flag of the variable indicates whether the last thread commits the transaction
    public volatile static boolean flag = true;

    public static void main(String[] args) throws InterruptedException {


        // The number of open threads
        int threadNum = 3;
        CountDownLatch main = new CountDownLatch(1);
        CountDownLatch child = new CountDownLatch(threadNum);
        ExecutorService executorPool = Executors.newFixedThreadPool(threadNum);

        for (int i = 1; i <= threadNum; i++) {
            final int n = i;
            executorPool.execute(() -> {
                try {

                    // Start the transaction
                    System.out.println(Thread.currentThread().getName() + "Performing" + "Open transaction");
                    // Simulate database operation to insert data
                    HashMap hashMap = new HashMap<String, String>();
                    hashMap.put("a3"."aa");

                    If (n == 2) {int i1 = 1/0; } * /
                    System.out.println(Thread.currentThread().getName() + "Performing" + "Waiting to commit transaction");
                    // Wait for the main thread to respond
                    child.countDown();
                    // Wait for the main thread to complete
                    main.await();
                    // Rollback operation
                    if(! flag) { System.out.println(Thread.currentThread().getName() +"Performing" + "The dao rollback");
                    } else {
                        // Commit transaction completed
                        System.out.println(Thread.currentThread().getName() + "Performing" + "Dao Commit transaction"); }}catch (Exception e) {
                    // Rollback operation
                    System.out.println(Thread.currentThread().getName() + "Exception found and execution started" + "The dao rollback");
                    flag = false; child.countDown(); }}); } child.await();if (flag) {
            System.out.println("Logical completion");
        } else {
            System.out.println("Logical failure");
        }
        main.countDown();
        executorPool.shutdown();
        System.out.println("ok"); }}Copy the code

Create a fixed 3 threads to execute our import logic in batches, each thread is divided into 10W +, and then use programtic transaction management to manage transaction behavior, wait for the response of the main thread after the operation, and then commit and roll back the transaction according to the flag variable. The main use of JUC tool class CountDownLatch to achieve.

Simulate an exception execution

Let’s execute and see what happens. All three threads are logically rolled back.  

Simulate the results of normal execution

Remove exception code. Transactions are committed normally.  

disadvantages

The above implementation will lose some data if, in the middle of execution, two threads commit transactions and suddenly the project dies. While performance and flexibility are greatly improved, they are less secure than single threads. In addition, do not create too many threads, which will occupy resources.

conclusion

This example also helps to understand the idea of distributed transactions, which are between processes, in this case between threads.

Reference: juejin. Cn/post / 689298…