View refresh (scheduleTraversals()) from ViewRootImpl I was asked about the Android screen refresh mechanism, double buffering, triple buffering and butter plan when I was interviewed by netease Cloud.
Screen refresh this set, you put me this article in the content of the clear, sure OK. Netease Cloud also asked me how the CPU and GPU exchange drawing data, which I personally think is a complete bonus question, I can not answer, interested friends can have a look, if you can clarify, I will definitely impress the interviewer.
double-buffering
Before we get into the concept of double buffering, let’s get to the basics.
Fundamentals of Display System
In a typical Display system, there are generally three parts: CPU, GPU and Display. CPU is responsible for calculating frame data and handing the calculated data to GPU, which will render the graphics data and store it in buffer(image buffer) after rendering. Then the Display(screen or Display) is responsible for rendering the data in the buffer onto the screen.
- Torn picture
The screen refresh frequency is fixed, for example, every 16.6ms to display a frame from the buffer. Ideally, the frame rate and refresh frequency are consistent, that is, every frame drawn, the monitor displays a frame. However, CPU/GPU write data is not controllable, so some data in the buffer will be overwritten before it is displayed at all, that is, the data in the buffer may be from different frames. When the screen refreshes, it does not know the state of the buffer, so the frame captured from the buffer is not a complete frame, that is, the frame is torn.
To put it simply, during the Display process, the data in the buffer is modified by CPU/GPU, causing the picture to tear.
So how do you fix the tear? The answer is to use double buffering.
double-buffering
Since the same buffer is used to draw the image and read the screen, it is possible to read an incomplete frame when the screen is refreshed.
Double buffering, so that drawing and display have their own buffer: The GPU always writes a completed Frame of image data to the Back Buffer, whereas the display uses the Frame Buffer. The Frame Buffer does not change when the screen refreshes, and they swap when the Back Buffer is ready.
VSync
When do I swap the two buffers?
If the Back buffer is intended to run after a frame has been completed, it will cause problems if the screen has not fully displayed the previous frame. It seems that you can only do this after the screen has processed a frame of data.
When a screen is scanned, the device needs to go back to the first row to enter the next cycle, with a time gap called the VerticalBlanking Interval(VBI). This point in time is the best time for us to swap buffers. Because the screen is not refreshing at this point, the screen is not torn during the exchange.
VSync, short for Vertical synchronization, uses the VerticalSync Pulse that emerged during the VBI era to ensure that dual buffers are exchanged at the optimal point in time. In addition, the swap refers to the respective memory address, which can be considered instantaneous.
So The concept of VSync was not invented by Google. It has been around since the early days of the PC industry.
Android screen refresh mechanism
In order to solve this problem, Google proposed Project Butter in Android4.1. The concept of drawing with VSync was introduced.
Jank (drop frames)
Take a look at what will happen in chronological order:
- Display displays the data of frame 0. At this time, CPU and GPU render the picture of frame 1 and finish it before Display displays the next frame
- Because rendering is timely, after Display is finished on frame 0, after VSync 1, the cache is swapped and then frame 1 is displayed normally
- Frame 2 is then processed, not until the second VSync is about to arrive.
- When the second VSync comes, frame 1 is still displayed because frame 2 is not ready and the cache is not swapped. This situation, named “Jank” by the Android development team, occurs when frames are lost.
- When frame 2 data is ready, it will not be displayed immediately, but will wait for the next VSync to cache swap.
So basically, the screen is showing frame 1 one more time for no reason. The reason is that the CPU/GPU calculation for frame 2 was not completed before the VSync signal arrived.
Notice one detail here, jank (drop frame, drop frame), not that this frame is dropped and not displayed, but that this frame is displayed late because the cache swap can only wait for the next VSync.
Butter plan — Drawing with VSync
In order to optimize Display performance, Google has reconfigured the Android Display system in Android 4.1 to implement Project Butter: The system will immediately start rendering the next frame after receiving the VSync Pulse. As soon as the VSync notification is received (triggered every 16ms), the CPU and GPU immediately start computing and write data to buffer. The diagram below:
The CPU/GPU synchronizes data processing according to VSYNC signal, allowing the CPU/GPU to have a complete 16ms time to process data, reducing jank. In summary, VSync synchronization allows the CPU/GPU to take full advantage of 16.6ms time, reducing jank.
Again, what if the interface is complex and the CPU/GPU processing time is longer than 16.6ms? The diagram below:
- In the second period, the GPU was still processing B frame, so the cache could not be exchanged, resulting in the repeated display of A frame.
- After B completes, it has to wait for the next signal because it lacks the VSync pulse signal. So in the process, a lot of time is wasted.
- When the next VSync occurs, the CPU/GPU immediately performs the operation (frame A), and the cache is swapped, and the corresponding display corresponds to B. It seems normal at this point. Just because of the execution time is still more than 16 ms, lead to the next should perform the buffer exchange was delayed, so repeated again, then there are more and more “Jank”.
Why can’t the CPU handle the drawing work in the second 16ms? Since there are only two buffers, the Back buffer is being used by THE GPU to process the data of Frame B, and the contents of the Frame buffer are used for Display Display. In this way, both buffers are occupied, and the CPU cannot prepare the data of the next Frame. Then, if another buffer is provided, the CPU, GPU, and display device can all use their own buffer without affecting each other. That’s where the triple buffer comes in.
Three buffer
Three-buffer is to add a Graphic Buffer on the basis of double Buffer mechanism, so as to maximize the use of idle time, the disadvantage of using more than one Graphic Buffer occupied memory.
- The first Jank was unavoidable. However, in the second 16ms period, CPU/GPU used the third Buffer to complete the calculation of C frame. Although A frame would still be displayed once more, the subsequent display would be relatively smooth, effectively avoiding the further aggravation of Jank.
- Note that in paragraph 3, the calculation of frame A is complete, but it is not shown until the fourth vsync comes, or if it is double-buffered, it is shown at the third VYNSC.
Triple buffering makes good use of the time spent waiting for VSync, reducing jank but introducing latency. Is the more buffers the better? This is negative. Two buffers are normal, but three are enough after Jank appears.
Choreographer
Now that’s the basics of refreshing, the real class for drawing on Android is called Choreographer.
Choreographer is responsible for the guidance of CPU/GPU drawing – only after receiving the VSync signal does the drawing begin, ensuring that the drawing has a full 16.6ms to avoid random drawing.
Typically, application layers will not use Choreographer directly, but rather more advanced apis such as animation and view-drawing related valueanimator.start (), view.invalidate (), etc.
Will onDraw be called back when property animations are updated? No, because it is internally updated through Choreographer in the AnimationHandler, the logic of which can be covered in a future article if there is time.
Choreographer is commonly used by the industry to monitor application frame rates.
(This is also an interview question, which asks how do you detect your app’s frame rate? You can mention FrameCallback in Choreographer and talk about it in combination with some third-party library implementations.
View refresh entry
The Activity starts, and after the onResume method completes, the window is added. The window addition procedure calls the ViewRootImpl setView() method, and the setView() method calls requestLayout() to request a layout, Inside the requestLayout() method comes the scheduleTraversals() method. This leads to performTraversals(), and then to the familiar three traversals of measurement, layout, and rendering.
When we use valueanimator.start (), view.invalidate (), we also end up with the ViewRootImpl scheduleTraversals() method. (View.invalidate() loops through the ViewParent until ViewRootImpl’s invalidateChildInParent() method, and then goes to scheduleTraversals().)
All UI changes go to the scheduleTraversals() method of ViewRootImpl.
One point to note here is that scheduleTraversals() is not immediately followed by performTraversals(), there is a Choreographer mechanism between them. To put it simply, in scheduleTraversals() Choreographer will request the native VSync signal and only when the VSync signal comes will the performTraversals() method be called for View rendering.
//ViewRootImpl.java
void scheduleTraversals(a) {
if(! mTraversalScheduled) { mTraversalScheduled =true;
// Add a synchronization barrier to mask synchronization messages and ensure that VSync is executed immediately
mTraversalBarrier = mHandler.getLooper().getQueue().postSyncBarrier();
//mTraversalRunnable is an instance of TraversalRunnable that leads to run(), i.e. doTraversal();
mChoreographer.postCallback(
Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null);
if (!mUnbufferedInputDispatch) {
scheduleConsumeBatchedInput();
}
notifyRendererOfFramePending();
pokeDrawLockIfNeeded();
}
}
final class TraversalRunnable implements Runnable {
@Override
public void run(a) { doTraversal(); }}final TraversalRunnable mTraversalRunnable = new TraversalRunnable();
void doTraversal(a) {
if (mTraversalScheduled) {
mTraversalScheduled = false;
// Remove the synchronization barriermHandler.getLooper().getQueue().removeSyncBarrier(mTraversalBarrier); .// Start three drawing processesperformTraversals(); . }}Copy the code- PostSyncBarrier Enables synchronization barriers to be drawn immediately after VSync arrives
- MChoreographer. PostCallback () method, which sends a callback can be carried in the next frame, TraversalRunnable – >doTraversal() – >performTraversals() – > draw the flow when the next VSync comes.
Choreographer
Initialize the
MChoreographer, are used within the constructor of ViewRootImpl Choreographer. GetInstance () to create.
Choreographer, like Looper, is thread singleton, using the ThreadLocal mechanism to ensure uniqueness. Because Choreographer sends messages internally through FrameHandler, initialization will determine whether the current thread is Looper or not, and throw exceptions if not.
public static Choreographer getInstance(a) {
return sThreadInstance.get();
}
private static final ThreadLocal<Choreographer> sThreadInstance =
new ThreadLocal<Choreographer>() {
@Override
protected Choreographer initialValue(a) {
Looper looper = Looper.myLooper();
if (looper == null) {
// For the current thread to have looper, Choreographer instances need to be passed in
throw new IllegalStateException("The current thread must have a looper!");
}
Choreographer choreographer = new Choreographer(looper, VSYNC_SOURCE_APP);
if (looper == Looper.getMainLooper()) {
mMainInstance = choreographer;
}
returnchoreographer; }};Copy the codepostCallback
MChoreographer. PostCallback (Choreographer. CALLBACK_TRAVERSAL mTraversalRunnable, null) method, the first parameter is the CALLBACK_TRAVERSAL, Indicates the type of the callback task. There are five types:
// Input events, executed first
public static final int CALLBACK_INPUT = 0;
// animation, second execution
public static final int CALLBACK_ANIMATION = 1;
// Insert updated animation, third execution
public static final int CALLBACK_INSETS_ANIMATION = 2;
// Draw, execute fourth
public static final int CALLBACK_TRAVERSAL = 3;
// Submit, execute,
public static final int CALLBACK_COMMIT = 4;
Copy the codeThe five types of tasks are stored in the corresponding CallbackQueue. Each time a VSYNC signal is received, Choreographer will first process INPUT tasks, then ANIMATION tasks, and finally TRAVERSAL tasks.
Internal call postCallbackDelayed postCallback () (), and then call postCallbackDelayedInternal (), normal execution scheduleFrameLocked message, Delayed messages send a meessage of type MSG_DO_SCHEDULE_CALLBACK:
private void postCallbackDelayedInternal(int callbackType,
Object action, Object token, long delayMillis) {...synchronized (mLock) {
...
mCallbackQueues[callbackType].addCallbackLocked(dueTime, action, token);
if (dueTime <= now) { // Execute immediately
scheduleFrameLocked(now);
} else {
ScheduleFrameLocked ()
Message msg = mHandler.obtainMessage(MSG_DO_SCHEDULE_CALLBACK, action);
msg.arg1 = callbackType;
msg.setAsynchronous(true); mHandler.sendMessageAtTime(msg, dueTime); }}}Copy the codeThe FrameHandler class is used internally to process messages. You can see that delayed MSG_DO_SCHEDULE_CALLBACK messages also end up at scheduleFrameLocked:
private final class FrameHandler extends Handler {
public FrameHandler(Looper looper) {
super(looper);
}
@Override
public void handleMessage(Message msg) {
switch (msg.what) {
case MSG_DO_FRAME:
// Perform doFrame, the drawing process
doFrame(System.nanoTime(), 0);
break;
case MSG_DO_SCHEDULE_VSYNC:
// Request a VSYNC signal, such as when you need to draw a task
doScheduleVsync();
break;
case MSG_DO_SCHEDULE_CALLBACK:
// Tasks that need to be delayed will eventually execute the above two events
doScheduleCallback(msg.arg1);
break; }}}void doScheduleCallback(int callbackType) {
synchronized (mLock) {
if(! mFrameScheduled) {final long now = SystemClock.uptimeMillis();
if(mCallbackQueues[callbackType].hasDueCallbacksLocked(now)) { scheduleFrameLocked(now); }}}}Copy the codeApply for VSync signal
The scheduleFrameLocked() method actually applies the VSync signal.
private void scheduleFrameLocked(long now) {
if(! mFrameScheduled) { mFrameScheduled =true;
if (USE_VSYNC) {
// The current thread of execution is the thread of mLooper
if (isRunningOnLooperThreadLocked()) {
// Request VSYNC signal
scheduleVsyncLocked();
} else {
// If not, mHandler is used to send the message to the original thread, and scheduleVsyncLocked is called again
Message msg = mHandler.obtainMessage(MSG_DO_SCHEDULE_VSYNC);
msg.setAsynchronous(true);/ / asynchronousmHandler.sendMessageAtFrontOfQueue(msg); }}else {
// If VSYNC is not enabled, the doFrame method is enabled directly.
final long nextFrameTime = Math.max(
mLastFrameTimeNanos / TimeUtils.NANOS_PER_MS + sFrameDelay, now);
Message msg = mHandler.obtainMessage(MSG_DO_FRAME);
msg.setAsynchronous(true);/ / asynchronousmHandler.sendMessageAtTime(msg, nextFrameTime); }}}Copy the codeThe registration and listening of VSync signals is implemented through mDisplayEventReceiver. MDisplayEventReceiver is created in the Choreographer’s constructor, is FrameDisplayEventReceiver instance. FrameDisplayEventReceiver is a subclass of DisplayEventReceiver,
private void scheduleVsyncLocked(a) {
mDisplayEventReceiver.scheduleVsync();
}
Copy the codepublic DisplayEventReceiver(Looper looper, int vsyncSource) {
if (looper == null) {
throw new IllegalArgumentException("looper must not be null");
}
mMessageQueue = looper.getQueue();
// Register native VSYNC signal listeners
mReceiverPtr = nativeInit(new WeakReference<DisplayEventReceiver>(this), mMessageQueue,vsyncSource);
mCloseGuard.open("dispose");
}
Copy the codeVSync signal callback
When the native VSync signal arrives, it goes to the onVsync() callback:
private final class FrameDisplayEventReceiver extends DisplayEventReceiver
implements Runnable {
@Override
public void onVsync(long timestampNanos, long physicalDisplayId, int frame) {...// Run (doFrame()); // Run (doFrame())
Message msg = Message.obtain(mHandler, this);
msg.setAsynchronous(true);
mHandler.sendMessageAtTime(msg, timestampNanos / TimeUtils.NANOS_PER_MS);
}
@Override
public void run(a) {
mHavePendingVsync = false; doFrame(mTimestampNanos, mFrame); }}Copy the code(Here’s another interview question: Will the onVsync() callback be executed when the UI is not refreshed? No, because VSync is automatically applied when the UI needs to be refreshed, not when the Native layer constantly pushes the callback up.
doFrame
The doFrame() method uses the doCallbacks() method to execute callbacks. The main thing is to take the queue of the corresponding task type and traverse the queue to execute all tasks, including the mTraversalRunnable rendering task initiated by ViewRootImpl. MTraversalRunnable executes the doTraversal() method, removes the synchronization barrier, and calls performTraversals() to begin the three rendering processes.
At this point the whole process is closed.
reference
Blog.csdn.net/litefish/ar…