Optimization analysis of MTK Camera cold start and back-and-forth switching performance

Lifelong Learning with you, this is Android programmer

This article covers some of the basics of Android development. Here’s what you’ll learn:

TraceView tutorial 5, Surface Create optimization 6, Systrace tutorial 7, Optimization plan 8, before and after switching speed optimization 9, optimization plan 10, hot start optimization 11, solution: 12, drive optimization

The background,

Compared with the cameras of redmi 6A and 6 on the same platform, our cameras of ID5A and ID6 platform have poor performance of hot/cold startup and front/back switching. Compared with competing products, we expect the optimized cameras to be 10% faster than those of Redmi 6A and 6.

Second, problem decomposition

The core of performance optimization is to find the hotspot that affects the performance. There are routines to find hotspot. First, the camera startup process should be decomposed and analyzed stage by stage.

The cold start of the camera can be decomposed into the following steps:

1.AMS starts the Activity of the camera -> The Activity of the camera receives the onCreated message (which can reflect the performance of the system). OnCreate begin -> onCreate end 3.openCamera -> CameraOpened 4.TextureView add -> SurfaceTexture created StartPreview -> PreviewStarted 6. SetWindow -> PreviewCallback

The Log is as follows:

01-01 12:01:57.099 794 3012 I ActivityManager: START u0 {act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10200000 CMP = com. Mediatek. Camera /. CameraLauncher BNDS = [150976] [290120] 2 (from the extras)} from the uid 10064-01 12:01:57. 01, 169, 8261 8261 D CamAp_QuickActivity: [0.000ms][BEGIN] onCreate 01-01 12:01:57.196 8261 8295 I CAMap_API1-handler-0: [openCamera]+ 01-01 12:01:57.217 532 2586 I mtkcam-dev1: [createSpecificCameraDevice1] dlopen libmtkcam_device1. So + 01-01 12:01:57. 224 532 2586 I mtkcam - dev1: [createSpecificCameraDevice1] dlopen libmtkcam_device1. So - 01-01 12:01:57. 259 532 2586 I mtkcam - dev1: [createSpecificCameraDevice1] xe6a0ec00 0 01-01 12:01:57. 260 532 2586 I mtkcam - dev1: 0[CameraDevice1Base::open] + 01-01 12:01:57.419 532 2586 I mtkcam-dev1: 0[CameraDevice1Base::open] Add new cameraId 0-0xe6A0EC00 01-01 12:01:57.419 532 2586 I mtkcam-dev1: 0[CameraDevice1Base::open] -01-01 12:01:57.424 8261 8295 I CAMap_API1-handler-0: [openCamera]-, Executing time = 228ms. 01-01 12:01:57.537 8261 8261 D CamAp_QuickActivity: [367.294ms] [END] onCreate 01-01 12:01:57.542 8261 8261 D CamAp_QuickActivity: [BEGIN] onStart 01-01 12:01:57.546 8261 8261 D CamAp_QuickActivity: [3.762ms] [END] onStart 01-01 12:01:57.549 8261 8261 D CamAp_QuickActivity: [BEGIN] onResume 01-01 12:01:57.552 8261 8261 D CamAp_QuickActivity: IsKeyguardLocked = false 01-01 12:01:57.552 8261 8261 D CamAp_QuickActivity: OnResume --> onPermissionResumeTasks() 01-01 12:01:57.600 8261 8261 D CamAp_QuickActivity: [END] onResume 01-01 12:01:57.685 8261 8294 I camap_API1-handler-0: [setPreviewDisplay]+, Pending time = 0ms. 01-01 12:01:57.686 8261 8294 I CAMap_API1-handler-0: [setPreviewDisplay]-, Executing time = 1ms. 01-01 12:01:57.700 8261 8294 I CAMap_API1-handler -0: [startPreview]+, Pending time = 0ms. 01-01 12:01:57.701 532 2586 W MTkcam-dev1: 0[CameraDevice1Base::initDisplayClient] NULL window is passed into... 01-01 12:01:57.701 532 2586 I mtkcam-dev1: [CameraDevice1Base: : startPreview] + 0 01-01 12:01:57. 720 8261 8261 I CamAp_TextureViewController: updatePreviewSize: new size (960 , 720 ) current size (0 , 0 ),mIsSurfaceCreated = false listener = com.mediatek.camera.common.mode.photo.PhotoMode$SurfaceChangeListener@a79ec89 01-01 12:01:57.735 532 2586 I mtkcam-dev1: 0 [CameraDevice1Base: : startPreview] - status (0) 01-01 12:01:57. I CamAp_API1 - Handler - 0, 736, 8261, 8294: [startPreview]-, Executing time = 36ms. 01-01 12:01:57.948 532 8363 I MtkCam/DCNode: [0::waitPreviewReady] Wait for start preview done + 01-01 12:01:58.087 532 8363 I MtkCam/DCNode: [0::waitPreviewReady]wait for start preview done - , Use 130 MS 01-01 12:01:58.053 8261 8261 D CamAp_TextureViewController: OnSurfaceTextureAvailable surface = android. Graphics. SurfaceTexture @ 2 d48d1c width 720 height 960-01 12:01:58. 01, 056, 8261  8294 I CamAp_API1-Handler-0: [setPreviewTexture]+, Pending time = 0ms. 01-01 12:01:58.087 532 8363 I MtkCam/CamAdapter: (8363)[transitState] StateIdle --> StatePreview 01-01 12:01:58.089 532 2586 I MtkCam/DisplayClient: [setWindow] + window(0xe6a0ecf0), WxH=960x720, count(6), INativeWindowConsumerUsage (256) 01-01 12:01:58. 093 8261 8294 I CamAp_API1 Handler - 0: [setPreviewTexture]-, Executing time = 75ms. 01-01 12:01:58.279 8261 8295 D CamAp_PhotoDeviceController: [onPreviewFrame] mModeDeviceCallback = com.mediatek.camera.common.mode.photo.PhotoMode@feb2273Copy the code

OnCreate begin -> OnCreate end: 367ms

01-01 12:01:57.169 8261 8261 D CamAp_QuickActivity: [0.000ms][BEGIN] onCreate 01-01 12:01:57.537 8261 8261 D CamAp_QuickActivity: [367.294ms] [END] onCreateCopy the code

OpenCamera: took 238ms


01-01 12:01:57.196  8261  8295 I CamAp_API1-Handler-0: [openCamera]+
01-01 12:01:57.424  8261  8295 I CamAp_API1-Handler-0: [openCamera]-, executing time = 228ms.

Copy the code

Surface Create: 333ms

01-01 12:01:57.720  8261  8261 I CamAp_TextureViewController: updatePreviewSize: new size (960 , 720 ) current size (0 , 0 ),mIsSurfaceCreated = false listener = com.mediatek.camera.common.mode.photo.PhotoMode$SurfaceChangeListener@a79ec89
01-01 12:01:58.053  8261  8261 D CamAp_TextureViewController: onSurfaceTextureAvailable surface  = android.graphics.SurfaceTexture@2d48d1c width 720 height 960

Copy the code

StartPreview: 386ms

01-01 12:01:57.701 532 2586 I mtkcam-dev1: [CameraDevice1Base: : startPreview] + 0 01-01 12:01:58. 087 532 8363 I MtkCam/CamAdapter: (8363)[transitState] StateIdle --> StatePreviewCopy the code

Where startWindow -> PreveiwCallback: spend 87ms

01-01 12:01:58.093  8261  8294 I CamAp_API1-Handler-0: [setPreviewTexture]-, executing time = 37ms.

01-01 12:01:58.279  8261  8295 D CamAp_PhotoDeviceController: [onPreviewFrame] mModeDeviceCallback = com.mediatek.camera.common.mode.photo.PhotoMode@feb2273

Copy the code

The steps that take time are 2, 3, 4, and 5. The main UI loads are openCamera, surfaceCreated, and startPreview. OpenCamera and startPreview take time to drive optimizations. UI loading and surfaceCreate need to be looked at at the top.

Third, tool analysis

TraceView analyzed the UI loading and found that there was no optimization. Now the loaded UI needs to be seen as soon as entering, and the view that does not need to be seen is delayed loading by the viewStub, and the UI has been optimized for several waves. The optimization space is relatively small. For a cold start, however, you can use a memory resident scheme, so that a cold start is actually a hot start. The main reason is that the camera will start a service to detect whether the temperature is too high when the phone starts up. The service pulls up the camera application. After memory is permanent, the application is not killed. This startup speed optimization is great. Current scheme: Projects above 2G use memory resident scheme by default.

TraceView tutorial

www.jianshu.com/p/f81b20b9a… Developer.android.com/studio/prof…

5. Surface Create optimization

The main focus now is Surface Create optimization. Systrace analyzed the surface creation process and found that ArcFilter HandleThread had been waiting.

Systrace tutorial

Developer. The android. Google. Cn/studio/comm…

Looking at the code, the filter is waiting for the ARC algorithm engine to initialize and the GL engine to initialize. The design here is controversial, in fact, when the camera starts up, the filter is not turned on, but the filter engine is initialized. Do take the benchmark:

01-02 01:22:07. 991, 23458-23554 /? I/CamAp_TextureEnvExt: handleGlEnvInit(): Enter 01-02 01:22:07.991 23458-23458/? V/CamAp_TextureEnvExt: getCamSurfaceTexture:: in-1 01-02 01:22:08.090 23458-23554/? I/CamAp_TextureEnvExt: handleGlEnvInit(): Exit with bRet = EGL_SUCCESS 01-02 01:22:08.091 23458-23554/? I/CamAp_TextureEnvExt: handleArcFilterEngine(): init begin 01-02 01:22:08.095 23458-23458/? V/CamAp_TextureEnvExt: getCamSurfaceTexture:: ID=1 01-02 01:22:08.211 23458-23554/? I/CamAp_TextureEnvExt: handleArcFilterEngine(): init end engine = 0,prepareEngine=0Copy the code

As you can see, it takes 99ms to initialize GLEnv and 120ms to initialize engine.

Seventh, optimization plan

There are two optimizations:

The filter engine is not loaded on startup and is loaded when switching to filter. The result is that when switching filters, the whole process needs to be modified, resulting in a large amount of code modification.
Step by step initialization of GLEnv and filter engine, theoretically optimized for 120ms+, creating Surface and startPreview for parallel execution. Code changes are small, and can achieve optimization effect.

The second option is currently used, code submission records:

http://192.168.10.10/#/c/213235/

Eight, before and after switching speed optimization

For the upper layer, there are mainly two aspects of switching speed, one is the flip animation, the other is gaussian blur. The flip animation is measured with a high-speed camera, only 9 frames, platform performance is limited, there is a feeling of immobility, the benchmarking machine does not have, communicate with the product to remove. What can be optimized is gaussian blur.

Gaussian blur time benchmark:

01-02 02:00:21. 636, 8089-8089 /? I/CamAp_GaussianBlur: GaussianBlur,create render:42 01-02 02:00:21.636 8089-8089/? D/CamAp_GaussianBlur: preview width 750 height 1500 01-02 02:00:21.731 8089-8089/? I/CamAp_GaussianBlur: getPreview bitmap Spent time: 125 01-02 02:00:21.821 8089-8089/? I/CamAp_GaussianBlur: blurBitmap,render bitmap:112Copy the code

The entire Gaussian blur process takes about 279ms, with getPreview and blurBitmap taking the main time, as well as init each time.

Ninth, optimization plan

Init costs 40 ms, just do it once. GetPreview, for Gaussian blur, we don’t need a clear image, we’re going to blur it anyway. In getPreview, we set a sampling rate, and the sampling quality is reduced by 16 times. In Gaussian blur, we set a smaller blur radius factor, which is currently 5, so that the blur speed is fast and the blur effect is good.

Optimized time benchmark:

01-02 02:14:15. 628, 20947-20947 /? D/CamAp_GaussianBlur: preview width 750 height 1500 01-02 02:14:15.632 20947-20947/? I/CamAp_GaussianBlur: getPreview bitmap Spent Time: 4 01-02 02:14:15.636 20947-20947/? I/CamAp_GaussianBlur: blurBitmap,render bitmap:4Copy the code

Code submission record:

http://192.168.10.10/#/c/213235/

10. Optimization of hot start

Hot start optimization, unlike contrast machine, before the preview came out, we made a Gaussian blur cover plate. It is easier to remove the cover plate when communicating with the product. Two problems arise that need to be solved.

TextureView is not destroyed and will keep the previous frame. It mainly shows that the hot start will flash on a frame.
The framework will have a screenshot screen that will also flash on frames.

Xi. Solution:

During pause, set the alpha of TextureView to 0, and on frame 1, set the alpha to 1.
App layer off screenshot.

Code submission record:

http://192.168.10.10/#/c/215036/ http://192.168.10.10/#/c/214142/

12. Drive optimization

For openCamera and startPreview optimizations, the bottom layer optimizes the 3A algorithm, adds fastae, and uses speed Mode mode. For details, please consult Peng Yebin.

The original link: www.jianshu.com/p/77d035588…

Friendly recommendation: Android dry goods to share

At this point, this has ended, if there is wrong place, welcome your suggestion and correction. Meanwhile, I look forward to your attention. Thank you for reading. Thank you!