Analysis of the principle of Choreographer screen refresh mechanism

[[439195]]

This article is reprinted from the WeChat public account "Android Development Programming", the author is Android Development Programming. Please contact the Android Development Programming public account for reprinting this article.

Preface

The Android platform provides two types of signals, one is a hardware signal and the other is a software signal, which is sent regularly by a thread of the SurfaceFlinger process. The hardware signal is sent by the hardware;

If the App process wants to use the GPU to draw images, it needs to receive the Vsync signal. Therefore, the App process accesses the SurfaceFlinger process to obtain this signal and then performs GPU drawing.

Android 4.1 and later added the Choreographer mechanism, which is used to coordinate with the Vsync mechanism to unify the timing of animation, input, and drawing.

Choreographer is the class responsible for obtaining the Vsync synchronization signal and controlling the App thread (main thread) to complete image drawing;

Today we are going to talk about the Choreographer mechanism;

1. Introduction to Choreographer Class

1. Instance initialization

 public ViewRootImpl(Context context, Display display) {
 ...
 //Get the Choreographer instance
    mChoreographer = Choreographer.getInstance();
 ...
 }
 public   static Choreographer getInstance() {
 return sThreadInstance.get();
 }
 private static final ThreadLocal<Choreographer> sThreadInstance =
        new ThreadLocal<Choreographer>() {
 @Override
    protected Choreographer initialValue() {
        Looper looper = Looper.myLooper();
        if (looper == null ) {
            throw new IllegalStateException( "The current thread must have a looper!" );
 }
 return new Choreographer(looper);
 }
 };

• A Choreographer instance object is saved in each thread;
• Thread local storage ThreadLocal variables, Choreographer type, when initializing variables in the main thread, create a Choreographer object and bind the main thread Looper;
• Each form of the same App uses the same Choregrapher object under ViewRootImpl, which controls the drawing rhythm of most views in the entire App.

2. Construction method

 private Choreographer(Looper looper, int vsyncSource) {
 mLooper = looper;
 //Use the current thread looper to create mHandler
        mHandler = new FrameHandler(looper);
 //USE_VSYNC 4.1 and above defaults to true , indicating the ability to accept VSync, which is FrameDisplayEventReceiver
        mDisplayEventReceiver = USE_VSYNC
                ? new FrameDisplayEventReceiver(looper, vsyncSource)
 : null ;
        mLastFrameTimeNanos = Long.MIN_VALUE;
 // Calculate the time for one frame. The refresh rate of the Android phone screen is 60Hz, which is 16ms
        mFrameIntervalNanos = (long)(1000000000 / getRefreshRate());
 // Create an array of linked list type CallbackQueue with a size of 5,
 //That is, there are five linked lists in the array, each of which stores the same type of tasks: input, animation, traversal drawing and other tasks (CALLBACK_INPUT, CALLBACK_ANIMATION, CALLBACK_TRAVERSAL)
        mCallbackQueues = new CallbackQueue[CALLBACK_LAST + 1];
 for ( int i = 0; i <= CALLBACK_LAST; i++) {
            mCallbackQueues[i] = new CallbackQueue();
 }
        // b/68769804: For low FPS experiments.
        setFPSDivisor(SystemProperties.getInt(ThreadedRenderer.DEBUG_FPS_DIVISOR, 1));
 }

• There is a Looper and a FrameHandler variable;
• The variable USE_VSYNC is used to indicate whether the system uses the Vsync synchronization mechanism. This value is obtained by reading the system property debug.choreographer.vsync;
• The system uses the Vsync synchronization mechanism and creates a FrameDisplayEventReceiver object to request and receive Vsync events;
• Choreographer creates a CallbackQueue array of size 3 to store different types of Callbacks.

3. Callback type

 //Input event, execute first
 public   static final int CALLBACK_INPUT = 0;
 //Animation, second execution
 public   static final int CALLBACK_ANIMATION = 1;
 //Insert updated animation, third execution
 public   static final int CALLBACK_INSETS_ANIMATION = 2;
 //Draw, fourth execution
 public   static final int CALLBACK_TRAVERSAL = 3;
 //Submit and execute finally,
 public   static final int CALLBACK_COMMIT = 4;

The five types of tasks are stored in the corresponding CallbackQueue;

Whenever a VSYNC signal is received, Choreographer will first process INPUT type tasks, then ANIMATION type, and finally TRAVELSAL type.

4. Messages processed by FrameHandler

• CallbackQueue is an array with a capacity of 4. Each element is used as a head pointer to lead to a linked list of the corresponding type. The four events are maintained through these four linked lists.
• FrameHandler mainly handles three types of messages:

 private final class FrameHandler extends Handler {
 public FrameHandler(Looper looper) {
 super(looper);
 }
 @Override
 public void handleMessage(Message msg) {
 switch (msg.what) {
 case MSG_DO_FRAME:
               doFrame(System.nanoTime(), 0);
 break;
 case MSG_DO_SCHEDULE_VSYNC:
 doScheduleVsync(); // Request VSYNC signal
 break;
 case MSG_DO_SCHEDULE_CALLBACK:
               doScheduleCallback(msg.arg1);
 break;
 }
 }
 }

• MSG_DO_FRAME handles the Runnable registered in Choreographer;
• MSG_DO_SCHEDULE_VSYNC directly requests the VSync signal of the next frame;
• MSG_DO_SCHEDULE_CALLBACK executes appropriate Handler delay processing according to Choreographer configuration;

2. Choreographer Execution Process

1. requestLayout

 @Override
 public void requestLayout() {
        if (!mHandlingLayoutInLayoutRequest) {
 checkThread(); //Check if it is in the current thread
 mLayoutRequested = true ; //mLayoutRequested whether to measure and layout.
            scheduleTraversals();
 }
 }
    void scheduleTraversals() {
 if (!mTraversalScheduled) {//Traversal will not be called multiple times in the same frame
            mTraversalScheduled = true ;
 mTraversalBarrier = mHandler.getLooper().getQueue().postSyncBarrier(); //Intercept synchronous Message
 //Choreographer callback, perform drawing operations
            mChoreographer.postCallback(
                    Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null );
 }
 }

• postSyncBarrier: Handler's synchronization barrier, which can intercept Looper's acquisition and distribution of synchronous messages. After adding the synchronization barrier, Looper will only obtain and process asynchronous messages. If there is no asynchronous message, it will enter a blocked state;
• Choreographer: Choreographer, unifying animation, input, and drawing timing;

 mChoreographer.postCallback(
                    Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null );
 postCallback()->postCallbackDelayed()->postCallbackDelayedInternal():

2. postCallbackDelayedInternal

 private void postCallbackDelayedInternal( int callbackType,
       Object action , Object token, long delayMillis) {
 synchronized (mLock) {
 // Current time
       final long now = SystemClock.uptimeMillis();
 //Callback execution time, which is the current time plus the delay time
       final long dueTime = now + delayMillis;
 // obtainCallbackLocked(long dueTime, Object action , Object token) will convert the three parameters passed in into CallbackRecord (see the source code for details, which is not the main part and is omitted here), and then CallbackQueue will add the CallbackRecord to the linked list according to the callback type.
       mCallbackQueues[callbackType].addCallbackLocked(dueTime, action , token);
 if (dueTime <= now) {
 // If delayMillis=0, dueTime=now, it will be executed immediately
           scheduleFrameLocked(now);
 } else {
 // If dueTime>now, send a scheduled message with what as MSG_DO_SCHEDULE_CALLBACK type, and process it when the time is up. The final processing is also to execute the scheduleFrameLocked(long now) method
           Message msg = mHandler.obtainMessage(MSG_DO_SCHEDULE_CALLBACK, action );
           msg.arg1 = callbackType;
           msg.setAsynchronous( true );
           mHandler.sendMessageAtTime(msg, dueTime);
 }
 }
 }

• mCallbackQueues first adds the corresponding callback to the linked list, and then determines whether there is a delay;
• If not, scheduleFrameLocked will be executed immediately. If so, a scheduled message with what is MSG_DO_SCHEDULE_CALLBACK type will be sent and processed when the time is up.
• Its final processing is also to execute the scheduleFrameLocked(long now) method;

3. scheduleFrameLocked

 private void scheduleFrameLocked(long now) {
 if (!mFrameScheduled) {
       mFrameScheduled = true ;
 if (USE_VSYNC) {
 // If VSYNC is used, it is determined by the system value
           if (DEBUG_FRAMES) {
               Log.d(TAG, "Scheduling next frame on vsync." );
 }
           if (isRunningOnLooperThreadLocked()) {
 // Request the VSYNC signal, which will eventually be transferred to the Native layer. After Native processing is completed, the onVsync callback of FrameDisplayEventReceiver will be triggered. The doFrame(long frameTimeNanos, int frame) method will also be called in the callback.
               scheduleVsyncLocked();
 } else {
 // Send a what=MSG_DO_SCHEDULE_VSYNC message directly on the UI thread, and eventually call scheduleVsyncLocked() to request the VSYNC signal
               Message msg = mHandler.obtainMessage(MSG_DO_SCHEDULE_VSYNC);
               msg.setAsynchronous( true );
               mHandler.sendMessageAtFrontOfQueue(msg);
 }
 } else {
 // Not using VSYNC
           final long nextFrameTime = Math. max (
                   mLastFrameTimeNanos / TimeUtils.NANOS_PER_MS + sFrameDelay, now);
           if (DEBUG_FRAMES) {
               Log.d(TAG, "Scheduling next frame in " + (nextFrameTime - now) + " ms." );
 }
 // Directly send a message with what=MSG_DO_FRAME, and call the doFrame(long frameTimeNanos, int frame) method when processing the message
           Message msg = mHandler.obtainMessage(MSG_DO_FRAME);
           msg.setAsynchronous( true );
           mHandler.sendMessageAtTime(msg, nextFrameTime);
 }
 }
 }

 // Enable/disable vsync for animations and drawing.
    private static final boolean USE_VSYNC = SystemProperties.getBoolean(
 "debug.choreographer.vsync" , true );

The constant USE_VSYNC indicates whether vertical synchronization of animation and drawing is allowed. The default value is true;

Determine USE_VSYNC. If VSYNC is used, scheduleVsyncLocked is used, which requests the VSYNC signal and finally calls doFrame.

If VSYNC is not used, doFrame is executed through the message;

4. scheduleVsyncLocked

The process of requesting VSYNC signal;

 private void scheduleVsyncLocked() {
    mDisplayEventReceiver.scheduleVsync();
 }
 public void scheduleVsync() {
 if (mReceiverPtr == 0) {
        Log.w(TAG, "Attempted to schedule a vertical sync pulse but the display event "  
                + "receiver has already been disposed." );
 } else {
        nativeScheduleVsync(mReceiverPtr);
 }
 }

• mDisplayEventReceiver corresponds to FrameDisplayEventReceiver, which inherits from DisplayEventReceiver and is mainly used to receive the synchronization pulse signal VSYNC;
• The scheduleVsync() method registers with the SurfaceFlinger service through the underlying nativeScheduleVsync(), that is, the dispatchVsync() method of DisplayEventReceiver will be called after the next pulse is received;
• This is similar to the subscriber mode, but each call to the nativeScheduleVsync() method has only one dispatchVsync() method callback;
• The bottom layer sends a VSYNC signal to the application layer, which is received by the Java layer through dispatchVsync(), and finally called back in onVsync of FrameDisplayEventReceiver;

5. FrameDisplayEventReceiver

 private final class FrameDisplayEventReceiver extends DisplayEventReceiver implements Runnable {
    private boolean mHavePendingVsync;
    private long mTimestampNanos;
 private int mFrame;
 @Override
 public void onVsync(long timestampNanos, int builtInDisplayId, int frame) {
 //Ignore Vsync from the second display
        if (builtInDisplayId != SurfaceControl.BUILT_IN_DISPLAY_ID_MAIN) {
 scheduleVsync();
 return ;
 }
 ...
        mTimestampNanos = timestampNanos;
 mFrame = frame;
 //The callback of this message is the current object FrameDisplayEventReceiver
        Message msg = Message.obtain(mHandler, this);
        msg.setAsynchronous( true );
 //Here mHandler is FrameHandler
        mHandler.sendMessageAtTime(msg, timestampNanos / TimeUtils.NANOS_PER_MS);
 }
 @Override
 public void run() {
        mHavePendingVsync = false ;
        doFrame(mTimestampNanos, mFrame);
 }
 }

• The onVsync() process sends a message with a built-in callback to the main thread Looper through FrameHandler, and the callback is FrameDisplayEventReceiver;
• When the main thread Looper executes the message, it calls the FrameDisplayEventReceiver.run() method, followed by doFrame;

6. doFrame

 void doFrame(long frameTimeNanos, int frame) {
        final long startNanos;
        synchronized (mLock) {
 ...
 //Is there any frame skipping? If so, print the log and correct the deviation
 }
 //Execute callback
 try {
            Trace.traceBegin(Trace.TRACE_TAG_VIEW, "Choreographer#doFrame" );
            AnimationUtils.lockAnimationClock(frameTimeNanos / TimeUtils.NANOS_PER_MS);
            mFrameInfo.markInputHandlingStart();
            doCallbacks(Choreographer.CALLBACK_INPUT, frameTimeNanos);
            mFrameInfo.markAnimationsStart();
            doCallbacks(Choreographer.CALLBACK_ANIMATION, frameTimeNanos);
            mFrameInfo.markPerformTraversalsStart();
            doCallbacks(Choreographer.CALLBACK_TRAVERSAL, frameTimeNanos);
            doCallbacks(Choreographer.CALLBACK_COMMIT, frameTimeNanos);
 finally
            AnimationUtils.unlockAnimationClock();
            Trace.traceEnd(Trace.TRACE_TAG_VIEW);
 }
 if (DEBUG_FRAMES) {
            final long endNanos = System.nanoTime();
            Log.d(TAG, "Frame " + frame + ": Finished, took "  
                    + (endNanos - startNanos) * 0.000001f + " ms, latency "  
                    + (startNanos - frameTimeNanos) * 0.000001f + " ms." );
 }
 }

The doFrame method renders the next frame, detects if there is a lag and fixes it, and then starts rendering. The parameters of the doCallbacks method are:

• CALLBACK_INPUT:input;
• CALLBACK_ANIMATION: animation;
• CALLBACK_TRAVERSAL: traverse, perform measure, layout, draw;
• CALLBACK_COMMIT: traverse the completed commit operations to correct the animation startup time;

7. doCallbacks

 void doCallbacks( int callbackType, long frameTimeNanos) {
 CallbackRecord callbacks;
 synchronized (mLock) {
        final long now = SystemClock.uptimeMillis();
 //Search for the CallbackRecord whose execution time has arrived from the CallbackQueue queue of the specified type
        callbacks = mCallbackQueues[callbackType].extractDueCallbacksLocked(now);
        if (callbacks == null ) {
 return ;
 }
        mCallbacksRunning = true ;
 }
 try {
 //Since CallbackQueues are sorted in chronological order, iterate over all the CallbackRecords that have arrived.
 for (CallbackRecord c = callbacks; c != null ; c = c. next ) {
            c.run(frameTimeNanos);
 }
 finally
        synchronized (mLock) {
            mCallbacksRunning = false ;
 do {
                final CallbackRecord next = callbacks.next ;
                recycleCallbackLocked(callbacks);
                callbacks = next ;
            } while (callbacks != null );
 }
 }
 }

Choreographer maintains these four linked lists internally. When rendering each frame, it will perform the corresponding rendering operations from top to bottom. If there is input, it will render the input queue first, if there is animation, it will render the animation, then traverse, and then submit;

8. Choreographer Summary

• Controlling external input event processing, animation execution, UI changes, and submission execution are all handled in the same class, namely Choreographer;
• Choreographer supports four types of events: input, drawing, animation, and submission, and registers through postCallback at the corresponding location where vsync needs to be synchronized for refresh, waiting for callback;
• Each time it is executed, Choreographer will only process the last event in the event list according to the current time. When there are time-consuming operations in the main thread, the event cannot be executed in time, which will cause the so-called "frame skipping" and "stuttering" phenomenon;
• Choreographer's common method postCallback(callbackType, Object) is a method for placing events into the event list, and doFrame() is a method for consuming these events;
• Choreographer monitors the underlying Vsync signal. Once a callback signal is received, it will call back the four types of events in the Java layer through doFrame.