Vsync signal mechanism and UI refresh process

[[439732]]

Preface

The screen refresh frame rate is unstable, the frame drops are serious, and 60 frames per second cannot be guaranteed, resulting in screen tearing;

Today we will explain the VSYNC mechanism and UI refresh process

1. Detailed explanation of Vsync signal

1. Knowledge points related to screen refresh

• Screen refresh rate: the number of times the screen is refreshed in one second (how many frames of images are displayed in one second), the unit is Hz (Hertz), such as the common 60 Hz. The refresh rate depends on the fixed parameters of the hardware (it will not change);
• Progressive scanning: The display does not display the image on the screen all at once, but scans from left to right and from top to bottom line by line, displaying the pixels of the entire screen in sequence, but this process is so fast that the human eye cannot detect the changes. Taking a 60 Hz refresh rate screen as an example, this process is 1000 / 60 ≈ 16ms;
• Frame rate: indicates the number of frames drawn by the GPU in one second, in fps. For example, in the film industry, 24 frames is enough to make the screen run very smoothly. The Android system uses a more streamlined 60 fps, which means that the GPU draws a maximum of 60 frames per second. The frame rate changes dynamically. For example, when the screen is still, the GPU has no drawing operations, and the screen refreshes the data in the buffer, that is, the frame data of the last operation of the GPU;
• Screen smoothness: running at 60 frames per second (16.6ms per frame), that is, 60fps, without any delay or frame drop;
• FPS: frames per second;
• Frame loss: The work was completed in 16.6ms but was not completed due to various reasons, which occupied the last n 16.6ms, equivalent to losing n frames;

2. VSYNC mechanism

VSync mechanism: The Android system sends out a VSYNC signal every 16ms to trigger the rendering of the UI. VSync is the abbreviation of Vertical Synchronization, which is a technology that has been widely used on PCs for a long time. It can be simply regarded as a timer interrupt. The VSync mechanism has been introduced in Android 4.1 (JB);

Drawing process under VSync mechanism; CPU/GPU receives vsync signal, Vsync is once every 16ms, so every time a Vsync command is issued, the CPU will perform a refresh operation. That is, at the first time of every 16ms, the CPU will respond to the Vsync command to refresh the data. The refresh time of the CPU and GPU is consistent with the FPS of the Display. Because only when the Vsync command is issued will the CPU and GPU perform refresh or display actions. The CPU/GPU receives the vsync signal to prepare the content to be displayed in the next frame in advance, so it can prepare the data of each frame in time to ensure the smoothness of the picture;

It can be seen that when the vsync signal does not remind the CPU/GPU to work, everything is normal within the first 16ms. However, within the second 16ms, the CPU calculates the data almost at the end of the time period and hands it over to the Graphics Driver, causing the GPU to draw at the end of the second period, and the entire action is delayed to the third period. This affects the drawing of the next screen. At this time, Jank (flickering, which can be understood as a freeze or pause) will appear. At this time, the CPU and GPU may be occupied by other operations, which is the reason for the freeze;

2. UI refresh principle process

1. VSYNC process diagram

When we change the content of TextView through setText, the UI interface will not change immediately. The App will first request the VSYNC service. After the next VSYNC signal is triggered, the UI of the App will really start to refresh. The basic process is as follows:

setText finally calls invalidate to apply for redrawing, and finally recursively calls invalidate of ViewRootImpl through ViewParent to request VSYNC. When requesting VSYNC, a synchronization fence will be added to prevent the execution of synchronization messages in the UI thread. This is done to speed up the response speed of VSYNC. If it is not set, a synchronization message is being executed when VSYNC arrives;

2. Invalidate view

View will recursively call invalidateChild of the parent container, backtracking level by level, and finally reach invalidate of ViewRootImpl.

 View.java​
 void invalidateInternal( int l, int t, int r, int b, boolean invalidateCache,
            boolean fullInvalidate) {
            // Propagate the damage rectangle to the parent view .
            final AttachInfo ai = mAttachInfo;
            final ViewParent p = mParent;
            if (p != null && ai != null && l < r && t < b) {
                final Rect damage = ai.mTmpInvalRect;
                damage.set (l, t, r, b);
                p.invalidateChild(this, damage);
 }
 ViewRootImpl.java
 void invalidate() {
    mDirty.set (0, 0, mWidth, mHeight);
 if (!mWillDrawSoon) {
 scheduleTraversals();
 }
 }

ViewRootImpl will call scheduleTraversals to prepare for redrawing. However, redrawing is generally not executed immediately. Instead, an mTraversalRunnable is added to the Choreographer.CALLBACK_TRAVERSAL queue of Choreographer, and VSYNC is applied at the same time. This mTraversalRunnable will not be executed until the requested VSYNC arrives.

3. Schedule Traversals

 ViewRootImpl.java
 // Add the mTraversalRunnable drawn by the UI to the callback waiting for the next vertical synchronization signal to arrive
 // mTraversalScheduled is used to ensure that before the current Traversals are executed, it will not require traversal on both sides, wasting 16ms, and there is no need to draw twice
 void scheduleTraversals() {
    if (!mTraversalScheduled) {
        mTraversalScheduled = true ;
 // Prevent synchronization fence, which means intercepting synchronization messages
        mTraversalBarrier = mHandler.getLooper().getQueue().postSyncBarrier();
 // When postCallback, request the vnsc vertical synchronization signal scheduleVsyncLocked
        mChoreographer.postCallback(
                Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null );
 <! --Add a callback to handle touch events to prevent Touch events from coming in the middle -->  
        if (!mUnbufferedInputDispatch) {
            scheduleConsumeBatchedInput();
 }
        notifyRendererOfFramePending();
        pokeDrawLockIfNeeded();
 }
 }

4. Apply for VSYNC synchronization signal

 Choreographer.java
 private void postCallbackDelayedInternal( int callbackType,
        Object action , Object token, long delayMillis) {
 synchronized (mLock) {
        final long now = SystemClock.uptimeMillis();
        final long dueTime = now + delayMillis;
        mCallbackQueues[callbackType].addCallbackLocked(dueTime, action , token);
 if (dueTime <= now) {
 <! --Apply for VSYNC synchronization signal-->  
            scheduleFrameLocked(now);
 }
 }
 }

5. scheduleFrameLocked

 // mFrameScheduled ensures that the vertical synchronization signal will only be requested once within 16ms
 // scheduleFrameLocked can be called multiple times, but mFrameScheduled ensures that no new requests will be issued before the next vsync arrives
 // Redundant scheduleFrameLocked calls are invalidated
 private void scheduleFrameLocked(long now) {
 if (!mFrameScheduled) {
        mFrameScheduled = true ;
 if (USE_VSYNC) {
            if (isRunningOnLooperThreadLocked()) {
                scheduleVsyncLocked();
 } else {
 // Because invalid already has a synchronization fence, mFrameScheduled is required so that the message can be executed by the UI thread
                Message msg = mHandler.obtainMessage(MSG_DO_SCHEDULE_VSYNC);
                msg.setAsynchronous( true );
                mHandler.sendMessageAtFrontOfQueue(msg);
 }
 }
 }
 }

• Before the currently requested VSYNC arrives, no new VSYNC will be requested, because it is meaningless to request two VSYNCs within 16ms;
• After VSYNC arrives, Choreographer uses Handler to encapsulate FrameDisplayEventReceiver into an asynchronous Message and sends it to the MessageQueue of the UI thread;

6. FrameDisplayEventReceiver

 private final class FrameDisplayEventReceiver extends DisplayEventReceiver
            implements Runnable {
        private boolean mHavePendingVsync;
        private long mTimestampNanos;
 private int mFrame;
 public FrameDisplayEventReceiver(Looper looper) {
 super(looper);
 }
 @Override
 public void onVsync(long timestampNanos, int builtInDisplayId, int frame) {
            long now = System.nanoTime();
            if (timestampNanos > now) {
 <! --Under normal circumstances, timestampNanos should not be greater than now. This is usually caused by a problem with the upload vsync mechanism. -->  
                timestampNanos = now;
 }
 <! --If the previous vsync synchronization signal is not executed, then the next one should not be responded to (it may be requested by other threads in some way)-->  
              if (mHavePendingVsync) {
                Log.w(TAG, "Already have a pending vsync event. There should only be "  
                        + "one at a time." );
 } else {
                mHavePendingVsync = true ;
 }
 <! --timestampNanos is actually the time when this vsync was generated, sent from the server-->  
            mTimestampNanos = timestampNanos;
 mFrame = frame;
            Message msg = Message.obtain(mHandler, this);
 <! --Since a synchronization fence already exists, the message arriving from VSYNC needs to be sent as an asynchronous message-->  
            msg.setAsynchronous( true );
            mHandler.sendMessageAtTime(msg, timestampNanos / TimeUtils.NANOS_PER_MS);
 }
 @Override
 public void run() {
            mHavePendingVsync = false ;
 <! --The mTimestampNanos here is actually the time when the Vynsc synchronization signal arrives, but when executing this message, there may be a delay-->  
            doFrame(mTimestampNanos, mFrame);
 }
 }

• The reason why it is encapsulated as an asynchronous Message is that a synchronization fence is added in front, and the synchronous message will not be executed;
• The UI thread is awakened, the message is taken out, and finally doFrame is called to refresh and redraw the UI;

7. doFrame

 void doFrame(long frameTimeNanos, int frame) {
 final long startNanos;
 synchronized (mLock) {
 <! --A lot of things have been done to ensure that there is a vertical synchronization signal every 16ms, an input, refresh, and redraw -->  
        if (!mFrameScheduled) {
 return ; // no   work   to do
 }
       long intendedFrameTimeNanos = frameTimeNanos;
        startNanos = System.nanoTime();
        final long jitterNanos = startNanos - frameTimeNanos;
 <! -- Check if the frame is dropped due to delayed execution. For every 16ms delay, one more frame is dropped. -->  
        if (jitterNanos >= mFrameIntervalNanos) {
            final long skippedFrames = jitterNanos / mFrameIntervalNanos;
 <! -- Skipped frames are actually the time that the last refresh request was delayed, but the fact that skippedFrames is 0 does not mean that there is no frame drop. -->  
            if (skippedFrames >= SKIPPED_FRAME_WARNING_LIMIT) {
 <! --If skippedFrames is too large, frames will definitely be dropped, but if it is 0, it does not mean that there are no frame drops -->  
                Log.i(TAG, "Skipped " + skippedFrames + " frames! "  
                        + "The application may be doing too much work on its main thread." );
 }
            final long lastFrameOffset = jitterNanos % mFrameIntervalNanos;
 <! --The real effective timestamp of starting doFrame-->  
            frameTimeNanos = startNanos - lastFrameOffset;
 }
        if (frameTimeNanos < mLastFrameTimeNanos) {
 <! --This situation usually means that there is a problem with the mechanism for generating vsync, so apply again-->  
            scheduleVsyncLocked();
 return ;
 }
 <! --intendedFrameTimeNanos is the timestamp that was originally drawn, frameTimeNanos is the real one, which can be used to identify the delay VSYNC in the rendering tool -->  
        mFrameInfo.setVsync(intendedFrameTimeNanos, frameTimeNanos);
 <! --Remove the mFrameScheduled judgment, indicating that the processing has started, -->  
        mFrameScheduled = false ;
        <! --Update mLastFrameTimeNanos-->  
        mLastFrameTimeNanos = frameTimeNanos;
 }
 try {
 <! --Really start doing business-->  
        Trace.traceBegin(Trace.TRACE_TAG_VIEW, "Choreographer#doFrame" );
 <! --Handle packaged move events-->  
        mFrameInfo.markInputHandlingStart();
        doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos);
 <! --Processing animation-->  
        mFrameInfo.markAnimationsStart();
        doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos);
 <! --Handle redrawing-->  
        mFrameInfo.markPerformTraversalsStart();
        doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos);
 <! --Submit->  
        doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos);
 finally
        Trace.traceEnd(Trace.TRACE_TAG_VIEW);
 }
 }

• doTraversal will first remove the fence, then process performTraversals, perform measurement, layout, drawing, and submit the current frame to SurfaceFlinger for layer synthesis display;
• The above multiple boolean variables ensure that the UI is redrawn at most once every 16ms;

9. Partial UI redraw

View redrawing and refreshing does not cause all Views to perform measurement, layout, and drawing. Only the View link to be refreshed needs to be adjusted, and the remaining Views may not need to waste energy to do it again;

 View.java​
 public RenderNode updateDisplayListIfDirty() {
        final RenderNode renderNode = mRenderNode;
 ...
        if ((mPrivateFlags & PFLAG_DRAWING_CACHE_VALID) == 0
                || !renderNode.isValid()
                || (mRecreateDisplayList)) {
 <! -- Invalid, need to redraw -->  
 } else {
 <! -- Still valid, no need to redraw -->  
            mPrivateFlags |= PFLAG_DRAWN | PFLAG_DRAWING_CACHE_VALID;
            mPrivateFlags &= ~PFLAG_DIRTY_MASK;
 }
 return renderNode;
 }

10. Draw the summary

• Android has a maximum of 60 FPS, which is determined by VSYNC, with a maximum of one frame every 16 ms;
• VSYNC will only be available if the client actively applies for it;
• It will be refreshed only when VSYNC arrives;
• If the UI does not change, VSYNC will not be requested and will not be refreshed;

Summarize

There are still many knowledge points about drawing, which will be summarized and released later;