Double buffering and segment locking
We need to start with Namenode’s double buffering mechanism, which has three flags
IsAutoSyncScheduled: Determines whether a cache swap is in progress
Synctxid: This is the transaction ID for the thread to flush disk
IsSyncRunning: Indicates whether the disk is being written
The analysis process
Thread 1 will first acquire the lock, then check to see if it is currently in the swap cache (via isAutoSyncScheduled), and if not, get a globally unique transaction ID, which is incremented. If thread 1 does not release the lock, thread 2, thread 3, and thread 4 will wait until thread 1 releases the lock. If thread 2, thread 3, and thread 4 acquire the lock, thread 1 will continue.
Let’s say that after thread 4 has finished writing and finds that bufCurrent has more than 512K of memory, the isAutoSyncScheduled Settings are set to true to indicate that it is time to swap and that no other thread can do the same. In this case, bufCurrent is the data written by threads 1,2,3,4.
If thread 4 has finished writing and finds that bufCurrent has exceeded 512K, isAutoSyncScheduled is set to true, indicating that it is time to swap memory
After the isAutoSyncScheduled configuration is set to true, thread 5 will have to wait and the bufCurrent memory will be empty. Set isSyncRunning to true. The isAutoSyncScheduled is then set to false and the wait thread is woken up.
After thread 4 wakes up the other threads, it starts writing bufReady data to disk. This operation is time-consuming, so it is not locked, but it is running, which is why it is marked green in the image below. BufReady is the data written by threads 1, 2, 3, and 4. At this point, thread 5 acquires the lock, realizes that the swap is complete, and starts writing data to bufCurrent.
If bufCurrent exceeds 512K again, isAutoSyncScheduled is set to true, indicating that you intend to swap memory, and other threads cannot write to bufCurrent. At this point thread 6 takes the lock in, but it finds that isAutoSyncScheduled is true, so it waits.
When thread 4 has finished writing to the disk and acquired the lock, it changes synctxID to its transaction ID and assigns isSyncRunning to false to indicate that the disk has been written. Finally wake up the wait thread 5. When thread 5 retakes the lock and finds isSyncRunning false, it continues swapping memory. After swapping, it changes isAutoSyncScheduled back to false to wake up thread 6, and then executes normally
So that’s double buffering and segment locking