View rendering process (6) Hardware rendering (1)

What steps does the ThreadedRenderer, the body object responsible for hardware rendering, take in the entire rendering process?

• 1. ThreadedRenderer enableHardwareAcceleration instantiated
• 2. The initialize initialization
• 3. Update updateSurface Surface
• ThreadedRenderer sets parameters like shadows
• 5. If you need to run invalidateRoot to check whether you need to search for invalid elements from the root
• 6. Draw Starts hardware rendering for View hierarchy drawing
• 7. UpdateDisplayListIfDirty update hardware rendering of the dirty area
• 8. Destroy hardware render objects

In the hardware rendering process, there is a core object named RenderNode, which serves as the node object for each View to draw.

Each time a drawing is prepared, the following three steps are performed:

• Rendernode. start generates a new DisplayListCanvas
• 2. Draw on the DisplayListCanvas such as calling the Drawable draw method and taking the DisplayListCanvas as the argument
• 3. Rendernode. end The RenderNode operation is complete

Important object

• 1.ThreadedRenderer manages all hardware rendering objects and is also the entry object for viewrotimPL for hardware rendering.
• RenderNode Each View will carry an object. When hardware rendering is enabled, the relevant rendering logic will be moved to RenderNode based on its judgment.
• Before each RenderNode actually starts to draw its own content, it needs to generate a DisplayListCanvas using the RenderNode. All drawing actions are drawn in the DisplayListCanvas. Finally, the DisplayListCanvas will be saved in the RenderNode.

In the Java layer for the Framework, there are only so many, and here is a schematic of the one-to-one mapping.

You can see that RenderNode actually builds along with the View tree and builds the entire display hierarchy. ThreadedRender is also able to use RenderNode as a cue to build a rendering process similar to software rendering.

Let me continue by introducing the core objects of the Native layer in hardware rendering.

• RenderNode is the root of all Rendernodes, from which all View hierarchy traversal begins. Similar to the duties of a DecorView in a View. But instead of corresponding to RootRenderNode, a DecorView has its own RenderNode.
• 2.RenderNode corresponds to native objects in the Java layer
• RenderThread Hardware rendering thread in which all rendering tasks are performed using the hardware RenderThread’s Looper.
• 4.CanvasContext is all the render context that will hold the PipeLine render PipeLine
• 5.PipeLine such as OpenGLPipeLine, SkiaOpenGLPipeLine, VulkanPipeLine rendering PipeLine. This rendering pipeline will perform the actual rendering behavior according to the Android system configuration
• 6.DrawFrameTask is the object in ThreadedRender that actually starts rendering
• RenderNodeProxy ThreadedRender native layer entry. It globally acts as RootRenderNode, CanvasContext, and RenderThread facade (facade design pattern).

Here’s a mind map:

ThreadedRenderer instantiation

The following function is called when mSurfaceHolder is found to be empty:

if (mSurfaceHolder == null) { enableHardwareAcceleration(attrs); . }Copy the code

This method calls the following method to create the ThreadedRenderer:

 mAttachInfo.mThreadedRenderer = ThreadedRenderer.create(mContext, translucent,
                        attrs.getTitle().toString());
Copy the code

public static boolean isAvailable() { if (sSupportsOpenGL ! = null) { return sSupportsOpenGL.booleanValue(); } if (SystemProperties.getInt("ro.kernel.qemu", 0) == 0) { sSupportsOpenGL = true; return true; } int qemu_gles = SystemProperties.getInt("qemu.gles", -1); if (qemu_gles == -1) { return false; } sSupportsOpenGL = qemu_gles > 0; return sSupportsOpenGL.booleanValue(); } public static ThreadedRenderer create(Context context, boolean translucent, String name) { ThreadedRenderer renderer = null; if (isAvailable()) { renderer = new ThreadedRenderer(context, translucent, name); } return renderer; }Copy the code

Whether or not the ThreadedRenderer can be created depends on the global configuration. If the value of ro.kernel.qemu is 0, OpenGL is supported and the value can be returned true. If qemu. Gles is -1, OpenGL ES does not support false and only software rendering can be used. Hardware rendering can only be turned on if qemu. Gles is set and greater than 0.

ThreadedRenderer constructor

ThreadedRenderer(Context context, boolean translucent, String name) { ... long rootNodePtr = nCreateRootRenderNode(); mRootNode = RenderNode.adopt(rootNodePtr); mRootNode.setClipToBounds(false); mIsOpaque = ! translucent; mNativeProxy = nCreateProxy(translucent, rootNodePtr); nSetName(mNativeProxy, name); ProcessInitializer.sInstance.init(context, mNativeProxy); loadSystemProperties(); }Copy the code

We can see that the ThreadedRenderer initializes and does three things:

• 1. NCreateRootRenderNode creates the RootRenderNode of the native layer, which is the root of all RenderNodes. The decorView-like role is the parent layout of all views. We treat the entire View hierarchy as a tree, so here is the root node.
• RenderNode. Adopt Creates a RenderNode at the root of the Java layer based on the native RootRenderNode.
• 3. NCreateProxy Specifies the proxy that creates the RenderNode. NSetName specifies the name of the proxy.
• 4. Initialize the GraphicsStats service for ProcessInitializer
• 5. LoadSystemProperties Reads the properties set by the system to the hardware renderer.

The key is to see what the ThreadRenderer does in points 1-3.

nCreateRootRenderNode

static jlong android_view_ThreadedRenderer_createRootRenderNode(JNIEnv* env, jobject clazz) {
    RootRenderNode* node = new RootRenderNode(env);
    node->incStrong(0);
    node->setName("RootRenderNode");
    return reinterpret_cast<jlong>(node);
}
Copy the code

You can see that this directly instantiates a RootRenderNode object and returns the address of the pointer directly.

class RootRenderNode : public RenderNode, ErrorHandler { public: explicit RootRenderNode(JNIEnv* env) : RenderNode() { mLooper = Looper::getForThread(); env->GetJavaVM(&mVm); }}Copy the code

You can see that RootRenderNode inherits the RenderNode object and holds a JavaVM, which is what we call a Java Virtual machine object. A Java process has only one JavaVM globally. The Looper object in ThreadLocal is also obtained using the getForThread method. This is actually the Looper of the UI thread.

Native layer RenderNode instantiation

RenderNode::RenderNode()
        : mDirtyPropertyFields(0)
        , mNeedsDisplayListSync(false)
        , mDisplayList(nullptr)
        , mStagingDisplayList(nullptr)
        , mAnimatorManager(*this)
        , mParentCount(0) {}
Copy the code

It’s important to have an mDisplayList in this constructor, remember it’s going to come up a lot. Let’s look at the RenderNode header:

class RenderNode : public VirtualLightRefBase { friend class TestUtils; // allow TestUtils to access syncDisplayList / syncProperties friend class FrameBuilder; public: .... ANDROID_API void setStagingDisplayList(DisplayList* newData); . bool isValid() { return mValid; } int getWidth() const { return properties().getWidth(); } int getHeight() const { return properties().getHeight(); }... AnimatorManager& animators() { return mAnimatorManager; }... const DisplayList* getDisplayList() const { return mDisplayList; } OffscreenBuffer* getLayer() const { return mLayer; } OffscreenBuffer** getLayerHandle() { return &mLayer; } // ugh... void setLayer(OffscreenBuffer* layer) { mLayer = layer; }... private: ... String8 mName; . uint32_t mDirtyPropertyFields; RenderProperties mProperties; RenderProperties mStagingProperties; bool mValid = false; DisplayList* mDisplayList; DisplayList* mStagingDisplayList; friend class AnimatorManager; AnimatorManager mAnimatorManager; OffscreenBuffer* mLayer = nullptr; std::vector<RenderNodeOp*> mProjectedNodes; . private: ... } /* namespace uirenderer */ } /* namespace android */Copy the code

I’m actually keeping a few important ones:

• 1. MDisplayList is actually all of the child RenderNode objects held in RenderNode
• 2. MStagingDisplayList This is generally a DisplayList that is saved after the View is iterated, and then converted to mDisplayList before drawing behavior
• RenderProperties mProperties is an object that stores information about the width and height of RenderNode
• 4.OffscreenBuffer mProperties RenderNode actually renders memory objects.

RenderNode.adopt

    public static RenderNode adopt(long nativePtr) {
        return new RenderNode(nativePtr);
    }
Copy the code

RenderNode, which wraps a Native layer, returns a Java layer object corresponding to the open Java layer operation API.

nCreateProxy

static jlong android_view_ThreadedRenderer_createProxy(JNIEnv* env, jobject clazz,
        jboolean translucent, jlong rootRenderNodePtr) {
    RootRenderNode* rootRenderNode = reinterpret_cast<RootRenderNode*>(rootRenderNodePtr);
    ContextFactoryImpl factory(rootRenderNode);
    return (jlong) new RenderProxy(translucent, rootRenderNode, &factory);
}
Copy the code

You can see that this process generates two objects:

• 1.ContextFactoryImpl Animation context factory

class ContextFactoryImpl : public IContextFactory {
public:
    explicit ContextFactoryImpl(RootRenderNode* rootNode) : mRootNode(rootNode) {}

    virtual AnimationContext* createAnimationContext(renderthread::TimeLord& clock) {
        return new AnimationContextBridge(clock, mRootNode);
    }

private:
    RootRenderNode* mRootNode;
};
Copy the code

This object actually lets RenderProxy hold a factory that creates the animation context. RenderProxy creates an AnimationContextBridge for each RenderNode using ContextFactoryImpl.

• 2.RenderProxy The proxy object of the root RenderNode. This proxy object will serve as the entry point for all draw-starting traversals.

RenderProxy Root Creation of a RenderNode proxy object

RenderProxy::RenderProxy(bool translucent, RenderNode* rootRenderNode,
                         IContextFactory* contextFactory)
        : mRenderThread(RenderThread::getInstance()), mContext(nullptr) {
    mContext = mRenderThread.queue().runSync([&]() -> CanvasContext* {
        return CanvasContext::create(mRenderThread, translucent, rootRenderNode, contextFactory);
    });
    mDrawFrameTask.setContext(&mRenderThread, mContext, rootRenderNode);
}
Copy the code

• RenderThread Hardware rendering thread through which all hardware rendering commands are queued for execution. The initialization method is as follows:

RenderThread::getInstance()
Copy the code

• 2.CanvasContext A hardware Canvas context that determines whether to use OpenGL ES or some other rendering pipe. The initialization method is as follows:

CanvasContext::create(mRenderThread, translucent, rootRenderNode, contextFactory);
Copy the code

• 2.DrawFrameTask The task object drawn for each frame.

Let’s take a look at what they do with their initialization in turn.

The initialization and running mechanism of RenderThread

RenderThread& RenderThread::getInstance() {
    static RenderThread* sInstance = new RenderThread();
    gHasRenderThreadInstance = true;
    return *sInstance;
}
Copy the code

You can see that it is simply a matter of calling the RenderThread constructor to instantiate and returning a pointer to the object.

RenderThread is a thread object. Let’s start with the object whose header inherits:

class RenderThread : private ThreadBase { PREVENT_COPY_AND_ASSIGN(RenderThread); public: // Sets a callback that fires before any RenderThread setup has occured. ANDROID_API static void setOnStartHook(void (*onStartHook)()); WorkQueue& queue() { return ThreadBase::queue(); }... }Copy the code

The queue of tasks queued in RenderThread is actually the WorkQueue object from ThreadBase.

class ThreadBase : protected Thread { PREVENT_COPY_AND_ASSIGN(ThreadBase); public: ThreadBase() : Thread(false) , mLooper(new Looper(false)) , mQueue([this]() { mLooper->wake(); }, mLock) {} WorkQueue& queue() { return mQueue; } void requestExit() { Thread::requestExit(); mLooper->wake(); } void start(const char* name = "ThreadBase") { Thread::run(name); }... }Copy the code

ThreadBase inherits from Thread objects. When the start method is called, the Thread’s run method is called to start the Thread.

A more critical object is to instantiate a Looper object into the WorkQueue. Instantiating Looper directly is actually creating a new Looper. But this Looper does not fetch the Looper of the leading thread. What does this Looper do? Find out below.

The WorkQueue sets a Looper method pointer to it, perhaps to wake up the Looper after completing a task.

RenderThread::RenderThread()
        : ThreadBase()
        , mVsyncSource(nullptr)
        , mVsyncRequested(false)
        , mFrameCallbackTaskPending(false)
        , mRenderState(nullptr)
        , mEglManager(nullptr)
        , mVkManager(nullptr) {
    Properties::load();
    start("RenderThread");
}
Copy the code

• 1. Read some global configuration from Properties and perform some configuration such as DEBUG.
• 2. Start Starts the current thread

The start method starts Thread’s run method. The run method ends up in the threadLoop method, and how it does so will be explained in the threads section of the virtual machine.

RenderThread::threadLoop

bool RenderThread::threadLoop() { setpriority(PRIO_PROCESS, 0, PRIORITY_DISPLAY); if (gOnStartHook) { gOnStartHook(); } initThreadLocals(); while (true) { waitForWork(); processQueue(); if (mPendingRegistrationFrameCallbacks.size() && ! mFrameCallbackTaskPending) { drainDisplayEventQueue(); mFrameCallbacks.insert(mPendingRegistrationFrameCallbacks.begin(), mPendingRegistrationFrameCallbacks.end()); mPendingRegistrationFrameCallbacks.clear(); requestVsync(); } if (! mFrameCallbackTaskPending && ! mVsyncRequested && mFrameCallbacks.size()) { requestVsync(); } } return false; }Copy the code

There are four key steps in threadloop:

• 1. InitThreadLocals initializes thread local variables
• 2. WaitForWork waits for RenderThread to render
• ProcessQueue performs the rendering work saved in WorkQueue
• 4. MPendingRegistrationFrameCallbacks greater than 0 or mFrameCallbacks greater than zero; And mFrameCallbackTaskPending to false, will call requestVsync, open the process of SF, EventThread blocking the listening to return. MFrameCallbackTaskPending represents the Vsync signal coming this method and perform the mFrameCallbackTaskPending to true.

initThreadLocals

void RenderThread::initThreadLocals() {
    mDisplayInfo = DeviceInfo::queryDisplayInfo();
    nsecs_t frameIntervalNanos = static_cast<nsecs_t>(1000000000 / mDisplayInfo.fps);
    mTimeLord.setFrameInterval(frameIntervalNanos);
    initializeDisplayEventReceiver();
    mEglManager = new EglManager(*this);
    mRenderState = new RenderState(*this);
    mVkManager = new VulkanManager(*this);
    mCacheManager = new CacheManager(mDisplayInfo);
}
Copy the code

Several core objects are created in this process:

• 1.EglManager When using OpenGL related pipes, the EglManager will perform context operations on OpenGL.
• VulkanManager When using Vulkan’s rendering pipeline, you will use VulkanManager for operations (Vulkan is a new generation OF 3D hardware graphics card rendering API, lighter and better performance than OpenGL).
• There are OpenGL and Vulkan pipes, Layer to render, etc.

Another core is initializeDisplayEventReceiver, the method for the WorkQueue which registered the listener:

void RenderThread::initializeDisplayEventReceiver() { LOG_ALWAYS_FATAL_IF(mVsyncSource, "Initializing a second DisplayEventReceiver?" ); if (! Properties::isolatedProcess) { auto receiver = std::make_unique<DisplayEventReceiver>(); status_t status = receiver->initCheck(); mLooper->addFd(receiver->getFd(), 0, Looper::EVENT_INPUT, RenderThread::displayEventReceiverCallback, this); mVsyncSource = new DisplayEventReceiverWrapper(std::move(receiver)); } else { mVsyncSource = new DummyVsyncSource(this); }}Copy the code

Registered in the stars can be seen on DisplayEventReceiver listening, namely Vsync signal monitoring, for displayEventReceiverCallback callback methods.

We temporarily first to RenderThread initializeDisplayEventReceiver method to explore here, we continue to see after the callback logic later.

WaitForWork blocks wait on objects that Looper listens on

    void waitForWork() {
        nsecs_t nextWakeup;
        {
            std::unique_lock lock{mLock};
            nextWakeup = mQueue.nextWakeup(lock);
        }
        int timeout = -1;
        if (nextWakeup < std::numeric_limits<nsecs_t>::max()) {
            timeout = ns2ms(nextWakeup - WorkQueue::clock::now());
            if (timeout < 0) timeout = 0;
        }
        int result = mLooper->pollOnce(timeout);
    }
Copy the code

You can see that the logic here is simple: it’s essentially calling Looper’s pollOnce method, blocking the loop in Looper until Vsync’s signal arrives.

processQueue

void processQueue() { mQueue.process(); }
Copy the code

The process method of the WorkQueue is actually called.

The process of WorkQueue

    void process() {
        auto now = clock::now();
        std::vector<WorkItem> toProcess;
        {
            std::unique_lock _lock{mLock};
            if (mWorkQueue.empty()) return;
            toProcess = std::move(mWorkQueue);
            auto moveBack = find_if(std::begin(toProcess), std::end(toProcess),
                                    [&now](WorkItem& item) { return item.runAt > now; });
            if (moveBack != std::end(toProcess)) {
                mWorkQueue.reserve(std::distance(moveBack, std::end(toProcess)) + 5);
                std::move(moveBack, std::end(toProcess), std::back_inserter(mWorkQueue));
                toProcess.erase(moveBack, std::end(toProcess));
            }
        }
        for (auto& item : toProcess) {
            item.work();
        }
    }
Copy the code

As you can see, this process is very simple and almost identical to the logic of the Message loop. If Looper’s block is turned on, the WorkItem whose expected execution time is greater than the current time is first found. It is removed from the mWorkQueue and finally added to the toProcess and executes the work method for each WorkItem. Each WorkItem is actually added to the mWorkQueue by a push-in method.

At this point, we understand how render tasks are consumed in RenderThread. So where do these render tasks come from?

RenderThread callback to the corresponding Vsync signal

Looper in the RenderThread listens for the Vsync signal and performs the following callbacks when the signal is called back.

displayEventReceiverCallback

int RenderThread::displayEventReceiverCallback(int fd, int events, void* data) { if (events & (Looper::EVENT_ERROR | Looper::EVENT_HANGUP)) { return 0; // remove the callback } if (! (events & Looper::EVENT_INPUT)) { return 1; // keep the callback } reinterpret_cast<RenderThread*>(data)->drainDisplayEventQueue(); return 1; // keep the callback }Copy the code

You can see that the core of this method is actually calling the drainDisplayEventQueue method to process the UI render task queue.

RenderThread::drainDisplayEventQueue

void RenderThread::drainDisplayEventQueue() { ATRACE_CALL(); nsecs_t vsyncEvent = mVsyncSource->latestVsyncEvent(); if (vsyncEvent > 0) { mVsyncRequested = false; if (mTimeLord.vsyncReceived(vsyncEvent) && ! mFrameCallbackTaskPending) { mFrameCallbackTaskPending = true; nsecs_t runAt = (vsyncEvent + DISPATCH_FRAME_CALLBACKS_DELAY); queue().postAt(runAt, [this]() { dispatchFrameCallbacks(); }); }}}Copy the code

Can I get to here mVsyncRequested set to false, and mFrameCallbackTaskPending will be set to true, and the queue is called postAt method performs the UI rendering method.

Remember that queue is actually a WorkQueue, and the postAt method of WorkQueue is actually implemented as follows:

template <class F> void postAt(nsecs_t time, F&& func) { enqueue(WorkItem{time, std::function<void()>(std::forward<F>(func))}); } void enqueue(WorkItem&& item) { bool needsWakeup; { std::unique_lock _lock{mLock}; auto insertAt = std::find_if( std::begin(mWorkQueue), std::end(mWorkQueue), [time = item.runAt](WorkItem & item) { return item.runAt > time; }); needsWakeup = std::begin(mWorkQueue) == insertAt; mWorkQueue.emplace(insertAt, std::move(item)); } if (needsWakeup) { mWakeFunc(); }}Copy the code

Case in point: when a Vsync signal reaches the Looper listener, it presses a task into the WorkQueue’s drainDisplayEventQueue.

Each of the default tasks executes the dispatchFrameCallback method. This checks if there is a time later than the current time in the mWorkQueue and returns the WorkItem. If the object in the header needsWakeup is true, it is ready to wake up. The mWakeFunc method pointer is passed from above:

mLooper->wake(); 
Copy the code

Wake up the blocked Looper. When awakened, the process method of the WorkQueue resumes execution. Which is to do the dispatchFrameCallbacks method.

RenderThread dispatchFrameCallbacks

void RenderThread::dispatchFrameCallbacks() { ATRACE_CALL(); mFrameCallbackTaskPending = false; std::set<IFrameCallback*> callbacks; mFrameCallbacks.swap(callbacks); if (callbacks.size()) { requestVsync(); for (std::set<IFrameCallback*>::iterator it = callbacks.begin(); it ! = callbacks.end(); it++) { (*it)->doFrame(); }}}Copy the code

Two things are done here:

• 1. RequestVsync enables listening blocking for EventThread.
• 2. Handle the doFrame method of IFrameCallback. Each IFrameCallback is added in the following way:

void RenderThread::pushBackFrameCallback(IFrameCallback* callback) { if (mFrameCallbacks.erase(callback)) { mPendingRegistrationFrameCallbacks.insert(callback); }}Copy the code

Added to the first mPendingRegistrationFrameCallbacks collection, in the above mentioned threadLoop, will perform the following logic:

if (mPendingRegistrationFrameCallbacks.size() && ! mFrameCallbackTaskPending) { drainDisplayEventQueue(); mFrameCallbacks.insert(mPendingRegistrationFrameCallbacks.begin(), mPendingRegistrationFrameCallbacks.end()); mPendingRegistrationFrameCallbacks.clear(); requestVsync(); }Copy the code

If mPendingRegistrationFrameCallbacks size not to 0, all the IFrameCallback mPendingRegistrationFrameCallbacks migration into mFrameCallbacks.

And when does this method get called? I’ll talk about that later. In fact, this part of the logic is mentioned in the TextureView parsing.

Initialization of CanvasContext

An important object will then be initialized:

CanvasContext::create(mRenderThread, translucent, rootRenderNode, contextFactory);
Copy the code

This object is called the context of the canvas. What context is it? Let’s look at the instantiation method now.

CanvasContext* CanvasContext::create(RenderThread& thread, bool translucent,
                                     RenderNode* rootRenderNode, IContextFactory* contextFactory) {
    auto renderType = Properties::getRenderPipelineType();

    switch (renderType) {
        case RenderPipelineType::OpenGL:
            return new CanvasContext(thread, translucent, rootRenderNode, contextFactory,
                                     std::make_unique<OpenGLPipeline>(thread));
        case RenderPipelineType::SkiaGL:
            return new CanvasContext(thread, translucent, rootRenderNode, contextFactory,
                                     std::make_unique<skiapipeline::SkiaOpenGLPipeline>(thread));
        case RenderPipelineType::SkiaVulkan:
            return new CanvasContext(thread, translucent, rootRenderNode, contextFactory,
                                     std::make_unique<skiapipeline::SkiaVulkanPipeline>(thread));
        default:
            break;
    }
    return nullptr;
}
Copy the code

on boot
    setprop debug.hwui.renderer opengl
Copy the code

The default in init.rc is OpengL, so let’s look at the following logic:

        case RenderPipelineType::OpenGL:
            return new CanvasContext(thread, translucent, rootRenderNode, contextFactory,
                                     std::make_unique<OpenGLPipeline>(thread));
Copy the code

An OpenGLPipeline is instantiated first, and then the OpenGLPipeline instantiates the CanvasContext as a parameter.

OpenGLPipeline instantiation

OpenGLPipeline::OpenGLPipeline(RenderThread& thread)
        : mEglManager(thread.eglManager()), mRenderThread(thread) {}
Copy the code

You can see that in the OpenGLPipeline, you actually store the RenderThread object and the mEglManager in the RenderThread. Through OpenGL pipeline to control mEglManager to further operate OpenGL.

CanvasContext instantiation

CanvasContext::CanvasContext(RenderThread& thread, bool translucent, RenderNode* rootRenderNode, IContextFactory* contextFactory, std::unique_ptr<IRenderPipeline> renderPipeline) : mRenderThread(thread) , mGenerationID(0) , mOpaque(! translucent) , mAnimationContext(contextFactory->createAnimationContext(mRenderThread.timeLord())) , mJankTracker(&thread.globalProfileData(), thread.mainDisplayInfo()) , mProfiler(mJankTracker.frames()) , mContentDrawBounds(0, 0, 0, 0) , mRenderPipeline(std::move(renderPipeline)) { rootRenderNode->makeRoot(); mRenderNodes.emplace_back(rootRenderNode); mRenderThread.renderState().registerCanvasContext(this); mProfiler.setDensity(mRenderThread.mainDisplayInfo().density); }Copy the code

The following operations are performed:

• RenderNode makeRoot method, adding a reference count for RenderNode
• Add the rootRenderNode rootRenderNode to the mRenderNodes collection. You can then locate the root RenderNode from mRenderNodes.
• 3. Get the renderState object of the RenderThread and call registerCanvasContext to save CanvasContext into mRegisteredContexts.

    void registerCanvasContext(renderthread::CanvasContext* context) {
        mRegisteredContexts.insert(context);
    }
Copy the code

MDrawFrameTask initialization

 mDrawFrameTask.setContext(&mRenderThread, mContext, rootRenderNode);
Copy the code

void DrawFrameTask::setContext(RenderThread* thread, CanvasContext* context,
                               RenderNode* targetNode) {
    mRenderThread = thread;
    mContext = context;
    mTargetNode = targetNode;
}
Copy the code

We’re essentially saving these three objects, the RenderThread; CanvasContext; RenderNode.

ThreadedRenderer nSetName

static void android_view_ThreadedRenderer_setName(JNIEnv* env, jobject clazz,
        jlong proxyPtr, jstring jname) {
    RenderProxy* proxy = reinterpret_cast<RenderProxy*>(proxyPtr);
    const char* name = env->GetStringUTFChars(jname, NULL);
    proxy->setName(name);
    env->ReleaseStringUTFChars(jname, name);
}
Copy the code

You can see that you’re actually calling the setName method of RenderProxy to set the name of the current hardware rendering object.

void RenderProxy::setName(const char* name) {
    mRenderThread.queue().runSync([this, name]() { mContext->setName(std::string(name)); });
}
Copy the code

You can see that in the setName method, we’re actually calling the WorkQueue of RenderThread, putting a task queue in it, and calling runSync to execute it.

    template <class F>
    auto runSync(F&& func) -> decltype(func()) {
        std::packaged_task<decltype(func())()> task{std::forward<F>(func)};
        post([&task]() { std::invoke(task); });
        return task.get_future().get();
    };
Copy the code

You can see that this method actually calls POST to queue the task, except that it uses the thread’s Future method, which blocks execution until the setName of the CanvasContext completes.

ThreadedRenderer initialization

boolean initialize(Surface surface) throws OutOfResourcesException { boolean status = ! mInitialized; mInitialized = true; updateEnabledState(surface); nInitialize(mNativeProxy, surface); return status; }Copy the code

The core is to call the nInitialize method and pass the Surface method to the threadedRenderer.

static void android_view_ThreadedRenderer_initialize(JNIEnv* env, jobject clazz,
        jlong proxyPtr, jobject jsurface) {
    RenderProxy* proxy = reinterpret_cast<RenderProxy*>(proxyPtr);
    sp<Surface> surface = android_view_Surface_getSurface(env, jsurface);
    proxy->initialize(surface);
}
Copy the code

The core here is simple: take the RenderProxy object and call initialize with the Native layer Surface object as an argument.

RenderProxy initialize

void RenderProxy::initialize(const sp<Surface>& surface) {
    mRenderThread.queue().post(
            [ this, surf = surface ]() mutable { mContext->setSurface(std::move(surf)); });
}
Copy the code

CanvasContext setSurface

void CanvasContext::setSurface(sp<Surface>&& surface) { mNativeSurface = std::move(surface); ColorMode colorMode = mWideColorGamut ? ColorMode::WideColorGamut : ColorMode::Srgb; bool hasSurface = mRenderPipeline->setSurface(mNativeSurface.get(), mSwapBehavior, colorMode); mFrameNumber = -1; if (hasSurface) { mHaveNewSurface = true; mSwapHistory.clear(); } else { mRenderThread.removeFrameCallback(this); mGenerationID++; }}Copy the code

In fact, the logic here is very simple, is to call setSurface to render pipeline (current OpenGLPipeline) method. If mSwapHistory is set successfully, clear it. If the mNativeSurface is null or invalid, the removeFrameCallback will be called to remove it, which is IFrameCallback.

bool OpenGLPipeline::setSurface(Surface* surface, SwapBehavior swapBehavior, ColorMode colorMode) { if (mEglSurface ! = EGL_NO_SURFACE) { mEglManager.destroySurface(mEglSurface); mEglSurface = EGL_NO_SURFACE; } if (surface) { const bool wideColorGamut = colorMode == ColorMode::WideColorGamut; mEglSurface = mEglManager.createSurface(surface, wideColorGamut); } if (mEglSurface ! = EGL_NO_SURFACE) { const bool preserveBuffer = (swapBehavior ! = SwapBehavior::kSwap_discardBuffer); mBufferPreserved = mEglManager.setPreserveBuffer(mEglSurface, preserveBuffer); return true; } return false; }Copy the code

Here the logic is very simple, go through the OpenGL initialization process, if the surface is not empty, then call mEglManager for the surface, create an OpenGL ES surface from the surface.

ThreadedRenderer updateSurface updates Surface

When the Surface changes in the Java layer, the updateSurface method is required to tell the hardware rendering thread to update the Surface method.

    void updateSurface(Surface surface) throws OutOfResourcesException {
        updateEnabledState(surface);
        nUpdateSurface(mNativeProxy, surface);
    }
Copy the code

ThreadedRenderer Setup starts hardware rendering and initializes parameters

void setup(int width, int height, AttachInfo attachInfo, Rect surfaceInsets) { mWidth = width; mHeight = height; if (surfaceInsets ! = null && (surfaceInsets.left ! = 0 || surfaceInsets.right ! = 0 || surfaceInsets.top ! = 0 || surfaceInsets.bottom ! = 0)) { mHasInsets = true; mInsetLeft = surfaceInsets.left; mInsetTop = surfaceInsets.top; mSurfaceWidth = width + mInsetLeft + surfaceInsets.right; mSurfaceHeight = height + mInsetTop + surfaceInsets.bottom; setOpaque(false); } else { mHasInsets = false; mInsetLeft = 0; mInsetTop = 0; mSurfaceWidth = width; mSurfaceHeight = height; } mRootNode.setLeftTopRightBottom(-mInsetLeft, -mInsetTop, mSurfaceWidth, mSurfaceHeight); nSetup(mNativeProxy, mLightRadius, mAmbientShadowAlpha, mSpotShadowAlpha); setLightCenter(attachInfo); }Copy the code

• 1. Call setLeftTopRightBottom of RootNode to update the left, right, and upper parameters of the RootNode.
• 2. NSetup ThreadedRenderer to load.

RootNode setLeftTopRightBottom

RootNode. SetLeftTopRightBottom method actually call is as follows:

static jboolean android_view_RenderNode_setLeftTopRightBottom(jlong renderNodePtr,
        int left, int top, int right, int bottom) {
    RenderNode* renderNode = reinterpret_cast<RenderNode*>(renderNodePtr);
    if (renderNode->mutateStagingProperties().setLeftTopRightBottom(left, top, right, bottom)) {
        renderNode->setPropertyFieldsDirty(RenderNode::X | RenderNode::Y);
        return true;
    }
    return false;
}
Copy the code

1. SetLeftTopRightBottom is the method used to set RenderProperties of RenderNode.

bool setLeftTopRightBottom(int left, int top, int right, int bottom) { if (left ! = mPrimitiveFields.mLeft || top ! = mPrimitiveFields.mTop || right ! = mPrimitiveFields.mRight || bottom ! = mPrimitiveFields.mBottom) { mPrimitiveFields.mLeft = left; mPrimitiveFields.mTop = top; mPrimitiveFields.mRight = right; mPrimitiveFields.mBottom = bottom; mPrimitiveFields.mWidth = mPrimitiveFields.mRight - mPrimitiveFields.mLeft; mPrimitiveFields.mHeight = mPrimitiveFields.mBottom - mPrimitiveFields.mTop; if (! mPrimitiveFields.mPivotExplicitlySet) { mPrimitiveFields.mMatrixOrPivotDirty = true; } return true; } return false; }Copy the code

You can see that you’re actually setting the parameters in mPrimitiveFields.

• 2. SetPropertyFieldsDirty tells the current RenderNode that it has changed.

void setPropertyFieldsDirty(uint32_t fields) { mDirtyPropertyFields |= fields; }
Copy the code

In the native ThreadedRenderer setup

static void android_view_ThreadedRenderer_setup(JNIEnv* env, jobject clazz, jlong proxyPtr,
        jfloat lightRadius, jint ambientShadowAlpha, jint spotShadowAlpha) {
    RenderProxy* proxy = reinterpret_cast<RenderProxy*>(proxyPtr);
    proxy->setup(lightRadius, ambientShadowAlpha, spotShadowAlpha);
}
Copy the code

We’re essentially calling the RenderProxy setup method.

RenderProxy setup

void RenderProxy::setup(float lightRadius, uint8_t ambientShadowAlpha, uint8_t spotShadowAlpha) {
    mRenderThread.queue().post(
            [=]() { mContext->setup(lightRadius, ambientShadowAlpha, spotShadowAlpha); });
}
Copy the code

You can see the setup that actually calls the CanvasContext.

void CanvasContext::setup(float lightRadius, uint8_t ambientShadowAlpha, uint8_t spotShadowAlpha) {
    mLightGeometry.radius = lightRadius;
    mLightInfo.ambientShadowAlpha = ambientShadowAlpha;
    mLightInfo.spotShadowAlpha = spotShadowAlpha;
}
Copy the code

And you can see that in this process you’re setting up some parameters for brightness and shadows.

ThreadedRenderer invalidateRoot opens the flag bit

First, ViewRootImpl’s draw method determines whether invalidateRoot needs to be called:

                if (invalidateRoot) {
                    mAttachInfo.mThreadedRenderer.invalidateRoot();
                }
Copy the code

    void invalidateRoot() {
        mRootNodeNeedsUpdate = true;
    }
Copy the code

You can see that it’s actually quite simple, the mRootNodeNeedsUpdate flag bit is set to true. The criterion of invalidateRoot is whether the displacement of the whole View tree changes. The whole is the entire displacement of the View tree. In the onMeasure, onLayout and onDraw processes of each View, determine whether it is software rendering or hardware rendering, and then determine whether to call the operation in RenderNode or Canvas operation in Skia.

ThreadedRenderer Draw starts the hardware rendering

Call timing in ViewRootImpl’s draw method:

 final FrameDrawingCallback callback = mNextRtFrameCallback;
                mNextRtFrameCallback = null;
                mAttachInfo.mThreadedRenderer.draw(mView, mAttachInfo, this, callback);
Copy the code

void draw(View view, AttachInfo attachInfo, DrawCallbacks callbacks, FrameDrawingCallback frameDrawingCallback) { attachInfo.mIgnoreDirtyState = true; final Choreographer choreographer = attachInfo.mViewRootImpl.mChoreographer; choreographer.mFrameInfo.markDrawStart(); updateRootDisplayList(view, callbacks); attachInfo.mIgnoreDirtyState = false; if (attachInfo.mPendingAnimatingRenderNodes ! = null) { final int count = attachInfo.mPendingAnimatingRenderNodes.size(); for (int i = 0; i < count; i++) { registerAnimatingRenderNode( attachInfo.mPendingAnimatingRenderNodes.get(i)); } attachInfo.mPendingAnimatingRenderNodes.clear(); attachInfo.mPendingAnimatingRenderNodes = null; } final long[] frameInfo = choreographer.mFrameInfo.mFrameInfo; if (frameDrawingCallback ! = null) { nSetFrameCallback(mNativeProxy, frameDrawingCallback); } int syncResult = nSyncAndDrawFrame(mNativeProxy, frameInfo, frameInfo.length); if ((syncResult & SYNC_LOST_SURFACE_REWARD_IF_FOUND) ! = 0) { setEnabled(false); attachInfo.mViewRootImpl.mSurface.release(); attachInfo.mViewRootImpl.invalidate(); } if ((syncResult & SYNC_INVALIDATE_REQUIRED) ! = 0) { attachInfo.mViewRootImpl.invalidate(); }}Copy the code

• 1. UpdateRootDisplayList traverses the entire View tree from the root.
• 2. RegisterAnimatingRenderNode RenderNode registered every need to perform the animation
• 3. NSetFrameCallback registers FrameDrawingCallback to the native layer
• 4. NSyncAndDrawFrame calls synchronous drawing of native
• 5. If the flag bit SYNC_LOST_SURFACE_REWARD_IF_FOUND is returned in the drawing result, the Surface may be invalid or an error may occur. In this case, close hardware rendering and release objects such as mSurface of VRI. If SYNC_INVALIDATE_REQUIRED returns, the next Loop will need to be refreshed locally via performTravel.

You can see that the whole core is in the updateRootDisplayList and nSyncAndDrawFrame methods. As long as you understand these two core processes, you can understand the entire hardware rendering process.

ThreadedRenderer updateRootDisplayList updates the draw tree from the root

private void updateRootDisplayList(View view, DrawCallbacks callbacks) { updateViewTreeDisplayList(view); if (mRootNodeNeedsUpdate || ! mRootNode.isValid()) { DisplayListCanvas canvas = mRootNode.start(mSurfaceWidth, mSurfaceHeight); try { final int saveCount = canvas.save(); canvas.translate(mInsetLeft, mInsetTop); callbacks.onPreDraw(canvas); canvas.insertReorderBarrier(); canvas.drawRenderNode(view.updateDisplayListIfDirty()); canvas.insertInorderBarrier(); callbacks.onPostDraw(canvas); canvas.restoreToCount(saveCount); mRootNodeNeedsUpdate = false; } finally { mRootNode.end(canvas); } } Trace.traceEnd(Trace.TRACE_TAG_VIEW); }Copy the code

The following things were done:

• 1. UpdateViewTreeDisplayList update the entire View tree
• 2. Call the start method of the root drawing node mRootNode to generate a root DisplayListCanvas
• 3. Call ViewRootImpl’s onPreDraw callback.
• 4.DisplayListCanvas insertReorderBarrier
• 5.DisplayListCanvas drawRenderNode draws the node and starts traversing the child nodes
• 6.DisplayListCanvas insertInorderBarrier
• 7. Call onPostDraw of ViewRootImpl
• 8. MRootNode end method, save the DisplayListCanvas into the root node.

Let’s do it one by one.

updateViewTreeDisplayList

    private void updateViewTreeDisplayList(View view) {
        view.mPrivateFlags |= View.PFLAG_DRAWN;
        view.mRecreateDisplayList = (view.mPrivateFlags & View.PFLAG_INVALIDATED)
                == View.PFLAG_INVALIDATED;
        view.mPrivateFlags &= ~View.PFLAG_INVALIDATED;
        view.updateDisplayListIfDirty();
        view.mRecreateDisplayList = false;
    }
Copy the code

Note that the View in this case refers to a DecorView, which sets the PFLAG_DRAWN flag bit to the DecorView.

If the PFLAG_INVALIDATED bit of the DecorView is turned on, the entire View is invalidated and the entire hardware-rendered View tree needs to be rebuilt. Then turn off the PFLAG_INVALIDATED bit and call updateDisplayListIfDirty for RenderNode traversal of the child View.

public RenderNode updateDisplayListIfDirty() { final RenderNode renderNode = mRenderNode; if (! canHaveDisplayList()) { return renderNode; }... mRecreateDisplayList = true; int width = mRight - mLeft; int height = mBottom - mTop; int layerType = getLayerType(); final DisplayListCanvas canvas = renderNode.start(width, height); try { if (layerType == LAYER_TYPE_SOFTWARE) { ... } else { ... canvas.translate(-mScrollX, -mScrollY); mPrivateFlags |= PFLAG_DRAWN | PFLAG_DRAWING_CACHE_VALID; mPrivateFlags &= ~PFLAG_DIRTY_MASK; if ((mPrivateFlags & PFLAG_SKIP_DRAW) == PFLAG_SKIP_DRAW) { dispatchDraw(canvas); drawAutofilledHighlight(canvas); if (mOverlay ! = null && ! mOverlay.isEmpty()) { mOverlay.getOverlayView().draw(canvas); } } else { draw(canvas); } } } finally { renderNode.end(canvas); setDisplayListProperties(renderNode); } } else { ... } return renderNode; }Copy the code

If this is a DecorView, the RenderNode object in the DecorView will be called as follows:

• Rendernode. start generates a corresponding DisplayListCanvas object for the DecorView.
• 2. Call the draw method and start drawing the DisplayListCanvas. During this process, the drawRenderNode method is also called.
• Rendernode. end saves the DisplayListCanvas object.

You can see that this is actually going back and forth calling the three RenderNode methods in each View hierarchy,

• 1. Start generates the View’s RenderNode
• 2. When the DisplayListCanvas draws the RenderNode, the DisplayListCanvas draws the RenderNode
• 3. Save DisplayListCanvas end

For those three methods, let’s take a look at the DisplayListCanvas methods.

DisplayListCanvas principle

Before we talk about DisplayListCanvas, we need to understand its inheritance in the Java layer. The diagram below:

RenderNode DisplayListCanvas generation

    public DisplayListCanvas start(int width, int height) {
        return DisplayListCanvas.obtain(this, width, height);
    }
Copy the code

    private static final int POOL_LIMIT = 25;
    private static final SynchronizedPool<DisplayListCanvas> sPool =
            new SynchronizedPool<>(POOL_LIMIT);

    static DisplayListCanvas obtain(@NonNull RenderNode node, int width, int height) {
        if (node == null) throw new IllegalArgumentException("node cannot be null");
        DisplayListCanvas canvas = sPool.acquire();
        if (canvas == null) {
            canvas = new DisplayListCanvas(node, width, height);
        } else {
            nResetDisplayListCanvas(canvas.mNativeCanvasWrapper, node.mNativeRenderNode,
                    width, height);
        }
        canvas.mNode = node;
        canvas.mWidth = width;
        canvas.mHeight = height;
        return canvas;
    }
Copy the code

You can see that in this process, all the DisplayListCanvas is fetched from a sPool. In fact, this is a free design pattern. There are up to 25 DisplayListCanvas in sPool. If you can’t get one, you just new it, otherwise you call nResetDisplayListCanvas to reset the DisplayListCanvas.

Let’s look at the DisplayListCanvas constructor again.

    private DisplayListCanvas(@NonNull RenderNode node, int width, int height) {
        super(nCreateDisplayListCanvas(node.mNativeRenderNode, width, height));
        mDensity = 0; 
    }

    @CriticalNative
    private static native long nCreateDisplayListCanvas(long node, int width, int height);
Copy the code

The native method nCreateDisplayListCanvas is first called to generate a native object and then the parent constructor is called.

Here are two interesting annotations that have been supported since Android 8.0: @criticalNative and @Fastnative.

These two annotations are annotations on native methods, which can speed up the search of native.

FastNative: Annotations support non-static methods (as well as static methods). Use this annotation if a method accesses Jobject as a parameter or return value. It’s three times faster than normal.

@CriticalNative is 5 times faster than normal, but the usage scenarios are limited:

• 1. Methods must be static – objects with no arguments, return values, or implicit this
• 2. Only primitive types are passed to native methods
• 3. Native methods do not use JNIEnv and jclass parameters in their function definitions
• 4. This method must be registered using RegisterNatives instead of relying on dynamic JNI links

nCreateDisplayListCanvas

static jlong android_view_DisplayListCanvas_createDisplayListCanvas(jlong renderNodePtr,
        jint width, jint height) {
    RenderNode* renderNode = reinterpret_cast<RenderNode*>(renderNodePtr);
    return reinterpret_cast<jlong>(Canvas::create_recording_canvas(width, height, renderNode));
}
Copy the code

You can see that the static method create_recording_canvas is actually called to create an object.

Canvas* Canvas::create_recording_canvas(int width, int height, uirenderer::RenderNode* renderNode) {
    if (uirenderer::Properties::isSkiaEnabled()) {
        return new uirenderer::skiapipeline::SkiaRecordingCanvas(renderNode, width, height);
    }
    return new uirenderer::RecordingCanvas(width, height);
}
Copy the code

During this process, a judgment is made that a SkiaRecordingCanvas will be created if the Skia switch is turned on, i.e., SKiaGL or SkiaVulan, otherwise.

The SkiaRecordingCanvas object, which I’ve talked about a little bit in TextureView, is now skia turned off by default.

The creation of RecordingCanvas

RecordingCanvas::RecordingCanvas(size_t width, size_t height)
        : mState(*this), mResourceCache(ResourceCache::getInstance()) {
    resetRecording(width, height);
}

void RecordingCanvas::resetRecording(int width, int height, RenderNode* node) {
    mDisplayList = new DisplayList();

    mState.initializeRecordingSaveStack(width, height);

    mDeferredBarrierType = DeferredBarrierType::InOrder;
}
Copy the code

RecordingCanvas, as its name implies, is a Canvas that records something.

• 1. First build a record drawing collection DisplayList object.
• 2. Call the current object initializeRecordingSaveStack method. That is called CanvasState initializeRecordingSaveStack.

void CanvasState::initializeRecordingSaveStack(int viewportWidth, int viewportHeight) { if (mWidth ! = viewportWidth || mHeight ! = viewportHeight) { mWidth = viewportWidth; mHeight = viewportHeight; mFirstSnapshot.initializeViewport(viewportWidth, viewportHeight); mCanvas.onViewportInitialized(); } freeAllSnapshots(); mSnapshot = allocSnapshot(&mFirstSnapshot, SaveFlags::MatrixClip); mSnapshot->setRelativeLightCenter(Vector3()); mSaveCount = 1; }Copy the code

If the width height is inconsistent with the currently instantiated swap height, initialize the mFirstSnapshot snapshot object and call the onViewportInitialized callback of RecordCanvas.

Release all snapshots except the first one and apply for a new snapshot object.

DisplayList instantiation

You can see that in the RecordingCanvas, there’s a core object DisplayList. As the name suggests, it is a collection of display layers.

class DisplayList {
    friend class RecordingCanvas;

public:
....

protected:
    // allocator into which all ops and LsaVector arrays allocated
    LinearAllocator allocator;
    LinearStdAllocator<void*> stdAllocator;

private:
    LsaVector<Chunk> chunks;
    LsaVector<BaseOpType*> ops;

    // list of Ops referring to RenderNode children for quick, non-drawing traversal
    LsaVector<NodeOpType*> children;

    // Resources - Skia objects + 9 patches referred to by this DisplayList
    LsaVector<sk_sp<Bitmap>> bitmapResources;
    LsaVector<const SkPath*> pathResources;
    LsaVector<const Res_png_9patch*> patchResources;
    LsaVector<std::unique_ptr<const SkPaint>> paints;
    LsaVector<std::unique_ptr<const SkRegion>> regions;
    LsaVector<sp<VirtualLightRefBase>> referenceHolders;

    // List of functors
    LsaVector<FunctorContainer> functors;

    LsaVector<VectorDrawableRoot*> vectorDrawables;

    void cleanupResources();
};
Copy the code

You can see that in the DisplayList class, there are several core objects:

• LinearAllocator and LinearStdAllocator. These two objects are memory managers used to control the BaseOpType memory of the application management operation object in hardware rendering.
• BaseOpType is the alias of the RecordedOp class, which refers to the drawing operation.
• 3.NodeOpType Set NodeOpType is the RenderNodeOp alias, which is used as a DisplayList to store the contents of child RenderNodes
• 4. The rest are bitmap resources,.9 image resources, brush words, regions, vector resources.

You can see that in the DisplayList, there are two objects that dominate the entire class:

• 1.BaseOpType draws operation objects
• 2.LinearAllocator BaseOpType Memory manager

Let’s look at the core principles of both.

LinearAllocator Linear memory manager

class LinearAllocator { public: template <class T> void* alloc(size_t size) { return allocImpl(size); } template <class T, typename... Params> T* create(Params&&... params) { T* ret = new (allocImpl(sizeof(T))) T(std::forward<Params>(params)...) ; if (! std::is_trivially_destructible<T>::value) { auto dtor = [](void* ret) { ((T*)ret)->~T(); }; addToDestructionList(dtor, ret); } return ret; } template <class T, typename... Params> T* create_trivial(Params&&... params) { static_assert(std::is_trivially_destructible<T>::value, "Error, called create_trivial on a non-trivial type"); return new (allocImpl(sizeof(T))) T(std::forward<Params>(params)...) ; } template <class T> T* create_trivial_array(int count) { static_assert(std::is_trivially_destructible<T>::value, "Error, called create_trivial_array on a non-trivial type"); return reinterpret_cast<T*>(allocImpl(sizeof(T) * count)); } private: LinearAllocator(const LinearAllocator& other); class Page; . void* allocImpl(size_t size); . size_t mPageSize; size_t mMaxAllocSize; void* mNext; Page* mCurrentPage; Page* mPages; . size_t mTotalAllocated; size_t mWastedSpace; size_t mPageCount; size_t mDedicatedPageCount; };Copy the code

You can see in this memory manager that each requested memory is actually managed in pages

• 1. MTotalAllocated tokens will also be used to keep track of how many tokens have been allocated
• 2. MWastedSpace records how much memory is not used
• 3. The mPages pointer is actually a linked list of pages that the current LinearAllocator has applied to all the memory pages.
• 4. MCurrentPage Page object to which the current application points.

Generally, when Android allocates memory, it will call in the following order:

create_trivial<TextureLayerOp>(
            Rect(layerHandle->getWidth(), layerHandle->getHeight()),
            *(mState.currentSnapshot()->transform), getRecordedClip(), layerHandle)
Copy the code

The create_trivial method is actually called.

template <class T, typename... Params> T* create_trivial(Params&&... params) { return new (allocImpl(sizeof(T))) T(std::forward<Params>(params)...) ; }Copy the code

We’re actually calling allocImpl based on the size of the norm T.

LinearAllocator memory unit Page

There is also a core object Page:

class LinearAllocator::Page {
public:
    Page* next() { return mNextPage; }
    void setNext(Page* next) { mNextPage = next; }
    Page() : mNextPage(0) {}
    void* operator new(size_t /*size*/, void* buf) { return buf; }
    void* start() { return (void*)(((size_t)this) + sizeof(Page)); }
    void* end(int pageSize) { return (void*)(((size_t)start()) + pageSize); }
private:
    Page(const Page& /*other*/) {}
    Page* mNextPage;
};
Copy the code

You can see that the Page class is actually an item in a linked list that records a pointer to a Page. Page has two basic operations to calculate the start and end of the requested memory for the page.

• Start refers to the starting point of the Page memory unit:

The value is Page address start point + Page class size

• End refers to the end of a Page memory unit:

The calculation method is Page address start point + Page class size + requested memory size PageSize

With two basic operations in mind, let’s look at the create_trivial memory operation on a LinearAllocator.

#define INITIAL_PAGE_SIZE ((size_t)512) // 512b #define MAX_PAGE_SIZE ((size_t)131072) // 128kb #if ALIGN_DOUBLE #define  ALIGN_SZ (sizeof(double)) #else #define ALIGN_SZ (sizeof(int)) #endif #define ALIGN(x) (((x) + ALIGN_SZ - 1) & ~ (ALIGN_SZ - 1)) # define ALIGN_PTR (p) ((void *) (ALIGN ((size_t) (p)))) # define MAX_WASTE_RATIO (0.5 f) LinearAllocator::LinearAllocator() : mPageSize(INITIAL_PAGE_SIZE) , mMaxAllocSize(INITIAL_PAGE_SIZE * MAX_WASTE_RATIO) , mNext(0) , mCurrentPage(0) , mPages(0) , mTotalAllocated(0) , mWastedSpace(0) , mPageCount(0) , mDedicatedPageCount(0) {} bool LinearAllocator::fitsInCurrentPage(size_t size) { return mNext && ((char*)mNext + size) <= end(mCurrentPage); } void* LinearAllocator::start(Page* p) { return ALIGN_PTR((size_t)p + sizeof(Page)); } void* LinearAllocator::end(Page* p) { return ((char*)p) + mPageSize; } LinearAllocator::Page* LinearAllocator::newPage(size_t pageSize) { pageSize = ALIGN(pageSize + sizeof(LinearAllocator::Page)); mTotalAllocated += pageSize; mPageCount++; void* buf = malloc(pageSize); return new (buf) Page(); }Copy the code

Let’s start with two important attributes:

• 1. MPageSize defines the current page size at initialization as INITIAL_PAGE_SIZE (macro 512B).
• 2. MMaxAllocSize = 512* 0.5 (256) Also refers to the maximum size that can be applied for, beyond this size needs to be expanded.
• 3. MNext records the end of the LinearAllocator application.

Let’s take a look at some important ways to find memory locations:

• 1. The start method actually offsets the current page pointer to the page size and ALIGN_PTR to the pointer address alignment. ALIGN_PTR operates by adding 3 to the current pointer address, followed by a non of 3, to ensure that the next two digits are zeros, which ensures that it is a multiple of 4.
• 2. End Calculates the end of the current memory. The size of the pointer + the size of the page requested
• 3. FitsInCurrentPage Determines whether the size of the current request can be satisfied. The calculation method is as follows: The starting point of the last request + the size to be applied < the end of the current request size
• 4. NewPage When the mMaxAllocSize threshold is exceeded, newPage will be called to apply for new memory. MTotalAllocated Allocated system – You can see how many pages mPageCount has allocated in mTotalAllocated system. The size of the application is requested by malloc:

New size = aligned memory (size of object to be applied + Page size)

AllocImpl application implementation

void* LinearAllocator::allocImpl(size_t size) { size = ALIGN(size); if (size > mMaxAllocSize && ! fitsInCurrentPage(size)) { Page* page = newPage(size); mDedicatedPageCount++; page->setNext(mPages); mPages = page; if (! mCurrentPage) mCurrentPage = mPages; return start(page); } ensureNext(size); void* ptr = mNext; mNext = ((char*)mNext) + size; mWastedSpace -= size; return ptr; }Copy the code

• 1. ALIGN the size to be applied with ALIGN bytes
• 2. Check whether the required size is larger than mMaxAllocSize and the size supported by the current page cannot meet the current size, apply a size page object through newPage, set the mNextPage object of page to the mPage object. And update the pointer to mPage to the latest object, finally call the start method, find the page out of the size of the pointer, and return the starting address.
• 3. If the size of the remaining Page is smaller than the size of mMaxAllocSize, use ensureNext to find the appropriate size and return the address starting from the mNext pointer. At the same time, record the new mNext address as the end of the application and wait for the next application. And reduce the size of mWastedSpace.

ensureNext

• 1. If fitsInCurrentPage determines that the size of the mNext Page is appropriate for the current Page, the current mNext Page will be returned.
• 2. If mCurrentPage exists and mPageSize(the size of applied pages) is smaller than MAX_PAGE_SIZE, the existing available pages cannot meet the required size and need to be expanded.

The MAX_PAGE_SIZE is 128kb

At the 128KB limit, each pair is expanded by twice the previous PageSize size. Update mMaxAllocSize to half of the new PageSize.

MPageSize needs to do one last bit of byte alignment.

• 3. MWastedSpace add a new mPageSize and apply for newPage according to the mPageSize. MCurrentPage if present, mNext of mCurrentPage is set to the newly applied page object. If mPages is empty, this is the first application. If mPages is empty, set mPages to mCurrentPage. MNext is set to the starting point of the current Page for new applications.

So here’s the memory management idea for LinearAllocator.

Each memory cell is managed as follows:

With this memory management support, the LinearAllocator destructor is called when destruction is needed.

LinearAllocator::~LinearAllocator(void) { while (mDtorList) { auto node = mDtorList; mDtorList = node->next; node->dtor(node->addr); } Page* p = mPages; while (p) { Page* next = p->next(); p->~Page(); free(p); RM_ALLOCATION(); p = next; }}Copy the code

RecordedOp Draws the underlying object for the operation

struct RecordedOp {
    /* ID from RecordedOpId - generally used for jumping into function tables */
    const int opId;

    /* bounds in *local* space, without accounting for DisplayList transformation, or stroke */
    const Rect unmappedBounds;

    /* transform in recording space (vs DisplayList origin) */
    const Matrix4 localMatrix;

    /* clip in recording space - nullptr if not clipped */
    const ClipBase* localClip;

    /* optional paint, stored in base object to simplify merging logic */
    const SkPaint* paint;

protected:
    RecordedOp(unsigned int opId, BASE_PARAMS)
            : opId(opId)
            , unmappedBounds(unmappedBounds)
            , localMatrix(localMatrix)
            , localClip(localClip)
            , paint(paint) {}
};
Copy the code

• 1. OpId RecordedOp Type ID of the rendering operation
• UnmappedBounds RecordedOp Area to draw (regardless of brush width and transform)
• 3. Transformation matrix of localMatrix coordinates
• 4. LocalClip Clipping area
• 5. Paint brush

Let’s take a look at a few classic inheritance examples: the action object LinesOp representing the draw line:

struct LinesOp : RecordedOp {
    LinesOp(BASE_PARAMS, const float* points, const int floatCount)
            : SUPER(LinesOp), points(points), floatCount(floatCount) {}
    const float* points;
    const int floatCount;
};
Copy the code

Line all points set points.

RecordedOp represents color manipulation

struct ColorOp : RecordedOp {
    // Note: unbounded op that will fillclip, so no bounds/matrix needed
    ColorOp(const ClipBase* localClip, int color, SkBlendMode mode)
            : RecordedOp(RecordedOpId::ColorOp, Rect(), Matrix4::identity(), localClip, nullptr)
            , color(color)
            , mode(mode) {}
    const int color;
    const SkBlendMode mode;
};
Copy the code

The color that represents the color value.

There are also TextureLayerOp objects that TextureView touches:

struct TextureLayerOp : RecordedOp {
    TextureLayerOp(BASE_PARAMS_PAINTLESS, DeferredLayerUpdater* layer)
            : SUPER_PAINTLESS(TextureLayerOp), layerHandle(layer) {}

    // Copy an existing TextureLayerOp, replacing the underlying matrix
    TextureLayerOp(const TextureLayerOp& op, const Matrix4& replacementMatrix)
            : RecordedOp(RecordedOpId::TextureLayerOp, op.unmappedBounds, replacementMatrix,
                         op.localClip, op.paint)
            , layerHandle(op.layerHandle) {}
    DeferredLayerUpdater* layerHandle;
};
Copy the code

The interior contains a DeferredLayerUpdater, due to updating a layer area of the DeferredLaydater object. For details on how DeferredLayerErupdater works, read my SurfaceView and TextureView source code (below), and note that TextView is the logic of Skia.

RenderNodeOp

In addition, there is another important object, RenderNodeOp:

struct RenderNodeOp : RecordedOp {
    RenderNodeOp(BASE_PARAMS_PAINTLESS, RenderNode* renderNode)
            : SUPER_PAINTLESS(RenderNodeOp), renderNode(renderNode) {}
    RenderNode* renderNode;  // not const, since drawing modifies it
    Matrix4 transformFromCompositingAncestor;
    bool skipInOrderDraw = false;
};
Copy the code

You can see that the RenderNodeOp operation actually contains a RenderNode object. In fact, the mDisplayList object will control all child RenderNodes through the RenderNodeOp.

size_t DisplayList::addChild(NodeOpType* op) {
    referenceHolders.push_back(op->renderNode);
    size_t index = children.size();
    children.push_back(op);
    return index;
}
Copy the code

Remember that this children is actually a collection of NodeOpTypes? You can see that the parent container’s RenderNode actually controls all child RenderNodes through the NodeOpType set of the DisplayList in its generated DisplayListCanvas.

The hardware rendering thread constructs the same View tree as the software rendering thread.

Canvas constructor

The RecordingCanvas constructor calls the base class constructor directly. Let’s look directly at the Canvas object:

    public Canvas(long nativeCanvas) {
        if (nativeCanvas == 0) {
            throw new IllegalStateException();
        }
        mNativeCanvasWrapper = nativeCanvas;
        mFinalizer = NoImagePreloadHolder.sRegistry.registerNativeAllocation(
                this, mNativeCanvasWrapper);
        mDensity = Bitmap.getDefaultDensity();
    }
Copy the code

It’s very simple to see that we’re actually listening for the collection method of mNativeCanvasWrapper.

DisplayListCanvas nResetDisplayListCanvas resets the entire DisplayListCanvas object

static void android_view_DisplayListCanvas_resetDisplayListCanvas(jlong canvasPtr,
        jlong renderNodePtr, jint width, jint height) {
    Canvas* canvas = reinterpret_cast<Canvas*>(canvasPtr);
    RenderNode* renderNode = reinterpret_cast<RenderNode*>(renderNodePtr);
    canvas->resetRecording(width, height, renderNode);
}
Copy the code

It’s actually calling the RecordingCanvas’s resetRecording, a new DisplayList, and the reset snapshot.

DisplayListCanvas drawRenderNode

    public void drawRenderNode(RenderNode renderNode) {
        nDrawRenderNode(mNativeCanvasWrapper, renderNode.getNativeDisplayList());
    }
Copy the code

static void android_view_DisplayListCanvas_drawRenderNode(jlong canvasPtr, jlong renderNodePtr) {
    Canvas* canvas = reinterpret_cast<Canvas*>(canvasPtr);
    RenderNode* renderNode = reinterpret_cast<RenderNode*>(renderNodePtr);
    canvas->drawRenderNode(renderNode);
}
Copy the code

It’s actually quite simple, just calling the drawRenderNode method of the RecordingCanvas.

RecordingCanvas drawRenderNode

void RecordingCanvas::drawRenderNode(RenderNode* renderNode) { auto&& stagingProps = renderNode->stagingProperties(); RenderNodeOp* op = alloc().create_trivial<RenderNodeOp>( Rect(stagingProps.getWidth(), stagingProps.getHeight()), *(mState.currentSnapshot()->transform), getRecordedClip(), renderNode); int opIndex = addOp(op); if (CC_LIKELY(opIndex >= 0)) { int childIndex = mDisplayList->addChild(op); // update the chunk's child indices DisplayList::Chunk& chunk = mDisplayList->chunks.back(); chunk.endChildIndex = childIndex + 1; if (renderNode->stagingProperties().isProjectionReceiver()) { // use staging property, since recording on UI thread mDisplayList->projectionReceiveIndex = opIndex; }}}Copy the code

The drawRenderNode method first adds the RenderNodeOp operation to the OPS operation set via addOp, and passes the child RenderNode to the current DisplayList for management via addChild.

RecordingCanvas addOp

int RecordingCanvas::addOp(RecordedOp* op) { // skip op with empty clip if (op->localClip && op->localClip->rect.isEmpty()) { return -1; } int insertIndex = mDisplayList->ops.size(); mDisplayList->ops.push_back(op); if (mDeferredBarrierType ! = DeferredBarrierType::None) { // op is first in new chunk mDisplayList->chunks.emplace_back(); DisplayList::Chunk& newChunk = mDisplayList->chunks.back(); newChunk.beginOpIndex = insertIndex; newChunk.endOpIndex = insertIndex + 1; newChunk.reorderChildren = (mDeferredBarrierType == DeferredBarrierType::OutOfOrder); newChunk.reorderClip = mDeferredBarrierClip; int nextChildIndex = mDisplayList->children.size(); newChunk.beginChildIndex = newChunk.endChildIndex = nextChildIndex; mDeferredBarrierType = DeferredBarrierType::None; } else { // standard case - append to existing chunk mDisplayList->chunks.back().endOpIndex = insertIndex + 1; } return insertIndex; }Copy the code

There are actually two operations:

• 1. The mDisplayList ops saves the current operation.
• 2. If mDeferredBarrierType is not None, obtain the chunk object at the end and save related parameters.

DisplayListCanvas end

RenderNode calls end after start and drawRenderNode:

    public void end(DisplayListCanvas canvas) {
        long displayList = canvas.finishRecording();
        nSetDisplayList(mNativeRenderNode, displayList);
        canvas.recycle();
    }
Copy the code

• 1. FinishRecord method ends the record drawn by Canvas. The method called is as follows:

DisplayList* RecordingCanvas::finishRecording() {
    restoreToCount(1);
    mPaintMap.clear();
    mRegionMap.clear();
    mPathMap.clear();
    DisplayList* displayList = mDisplayList;
    mDisplayList = nullptr;
    mSkiaCanvasProxy.reset(nullptr);
    return displayList;
}
Copy the code

You can see that you’re actually returning the DisplayList in the RecordingCanvas directly to the upper object.

• 2. NSetDisplayList RenderNode records the DisplayList.

static void android_view_RenderNode_setDisplayList(JNIEnv* env,
        jobject clazz, jlong renderNodePtr, jlong displayListPtr) {
    RenderNode* renderNode = reinterpret_cast<RenderNode*>(renderNodePtr);
    DisplayList* newData = reinterpret_cast<DisplayList*>(displayListPtr);
    renderNode->setStagingDisplayList(newData);
}
Copy the code

void RenderNode::setStagingDisplayList(DisplayList* displayList) { mValid = (displayList ! = nullptr); mNeedsDisplayListSync = true; delete mStagingDisplayList; mStagingDisplayList = displayList; }Copy the code

Native objects that can actually see RenderNode destroy the last DisplayList object and save the new DisplayList to mStagingDisplayList.

conclusion

This article stops here, so we know what the entire ThreadRenderer does to prepare for hardware rendering. We will then begin to parse the entire hardware rendering process. So let’s summarize.

Now take a look at the previous mind map:

RenderNode summary

Rendernodes can be divided into two types:

• 1. One is a RootRenderNode stored in the ThreadedRenderer, which is a RootRenderNode that is added to the CanvasContext at initialization and then traversed from the RootRenderNode for subsequent renderings.
• 2. The other is the object that follows the RenderNode of all views, which is the only object in all views. When hardware rendering is turned on, it goes to the RenderNode branch and puts all the drawing actions in the RenderNode.

For all RenderNodes, the following responsibilities apply:

DisplayListCanvas and DisplayList summary

All DisplayListCanvas is generated by RenderNode’s start method. The corresponding native object of DisplayListCanvas is the RecordingCanvas. Here is a more comprehensive UML diagram:

The following relationship can be seen from the figure:

• 1. DisplayListCanvas at the bottom and RecordingCanvas/SkiaRecordingCanvas one-to-one correspondence. All the content on the hardware render canvas is actually stored in the RecordingCanvas.
• 2.RecordingCanvas is called Recording for a simple reason, because it will not be drawn to memory immediately through draw, onDraw and drawdispatch. Instead, we break the draw operation into baseopTypes and store them in the DisplayList. Rendernode. end saves the DisplayList into RenderNode when a rendering is complete.
• 3. There is a key class DisplayList in the RecordingCanvas. DisplayList has two core sets that run through the global hardware rendering: OPS and children.

Ops is a set of BaseopTypes, which is actually a RecordedOp set. This collection generally does not record the RecordedOp infrastructure, but rather collects operations that have practical meaning through inheritance extended to Text, Color, etc.

The children are RenderNodeOp. RenderNodeOp contains child RenderNodes, although it also inherits RecordedOp, so hardware rendering does not store RenderNodeOp in the OPS collection, but in the Children collection during draw.

The summary of the RenderThread

Renderthreads are essentially hardware rendering threads that are active behind them:

                final FrameDrawingCallback callback = mNextRtFrameCallback;
                mNextRtFrameCallback = null;
                mAttachInfo.mThreadedRenderer.draw(mView, mAttachInfo, this, callback);
Copy the code

If the mThreadedRenderer is available, then the mThreadedRenderer. Draw will enter the RenderThread through the RenderProxy. Note that the RenderThread is a singleton rendering thread that contains a Looper and a WorkQueue. Its core principles are almost identical to MessageQueue and Looper.

All tasks are added to the WorkQueue via RenderThread. Looper retrieves workItems stored in the WorkQueue via epoll wake up. And these workItems contain Pointers to the methods we need to render.

Another thing that the Looper of RenderThread does in addition to executing the render methods saved in the WorkQueue is listen for Vsync signals to arrive. Every time a Vsync signal arrives, the dispatchFrameCallbacks method will be called to start calling back into the CanvasContext to perform the render action, which I’ll talk about in the next article.

Summary of CanvasContext and PipeLine

CanvasContext refers to the drawing context. CanvasContext holds a key object PipeLine. PipeLine will select OpenGL, SkIA-adapted OpenGL and SkIA-adapted Vulkan for hardware rendering according to the current configuration

In summary, all rendering behavior is centralized into the CanvasContext, and the CanvasContext decides which pipe to use for rendering behavior.

Each time the RenderThread listens for a Vsync signal, it calls back to the DrawFrameTask to perform the actual pixel composition.

The ThreadedRenderer’s role in view wrotimPL and execution timing

After summarizing the two core classes, let’s take a big look at the role ThreadedRenderer plays in view of the three main drawing processes in view WrotimPL. In fact, I skipped the hardware rendering process by accident in the previous few articles.

Perform the following steps in view otimpl’s performTraversals method:

• 1. EnableHardwareAcceleration instantiation ThreadedRenderer, at the same time build RootRenderNode, RenderProxy, CanvasContext, DrawFrameTask, etc. And start listening for Vsync signals through Looper.
• Initialize handles the WindowInset before calling onMeasure to traverse the global View tree. Enable render PipeLine to hold Surface objects so that it can communicate with the SF process via Surface.
• 3. UpdateSurface found Surface parameter changes after WindowInset processing before onMeasure traverses global View tree, updated Surface in hardware rendering PipeLine.
• When the Surface in the PipeLine is initialized and the ThreadedRenderer is started to set parameters such as shadows, the RootRenderNode left and top offsets are processed, i.e. the left and top offsets of the entire View tree.

The following steps determine whether hardware rendering can be done when the draw method is called:

• 5. If you need to execute invalidateRoot to determine whether you need to traverse from the root to find invalid elements, the judgment is based on whether the overall hardware rendering tree is offset

• 6. Draw Starts hardware rendering for View level drawing, which consists of the following steps: –

  • 1. UpdateRootDisplayList traverses the entire View tree from the root
  • 2. RegisterAnimatingRenderNode RenderNode registered every need to perform the animation
  • 3. NSetFrameCallback registers FrameDrawingCallback to the native layer
  • 4. NSyncAndDrawFrame calls native at the beginning of the draw operation to synthesize the pixel archive to memory.
  • 5. If the flag bit SYNC_LOST_SURFACE_REWARD_IF_FOUND is returned in the drawing result, the Surface may be invalid or an error may occur. In this case, close hardware rendering and release objects such as mSurface of VRI. If SYNC_INVALIDATE_REQUIRED returns, the next Loop will need to be refreshed locally via performTravel.

• 7. UpdateDisplayListIfDirty in updateRootDisplayList starting from the root tree traversal View in the process of every View calls this method to update the hardware rendering of the dirty area. When PFLAG_DRAWING_CACHE_VALID draws cache expiration flag bit off to indicate that there is no expiration, there is no need to do a draw, onDraw, dispatchDraw to save the draw operation to RenderNode’s DisplayList for compositing.
• 8. Destroy hardware render objects

Author: yjy239 links: www.jianshu.com/p/c84bfa909… The copyright of the book belongs to the author. Commercial reprint please contact the author for authorization, non-commercial reprint please indicate the source.