Author: Xianyu technology — Jingshu

background

The picture scheme we use for idle fish is our own external texture scheme:

  • Create the SurfaceTexture on Android, register it with the Flutter engine via FlutterJNI, and return the Texture ID to the Flutter application layer. The application layer uses the Texture Widget and textue ID to display the image textures.
  • Texture data is written to the SurfaceTexture via OpenGL on the Android side. The texture data is then passed to the application layer via shared memory in the Flutter Engine, and finally to Skia for rendering.

The problem with this is that the texture data of the Flutter application layer is not cached. Each time the Bitmap data needs to be re-rendered to the Flutter application layer for use. Native image loads are cached in memory. The image library provided by Flutter also has a cache. The two caches are isolated from each other and occupy a large amount of memory. In addition, the Flutter image cache is basically local resource images stored, while most of our Flutter pages are external texture images downloaded from the Internet, resulting in low utilization of cache resources.

Analysis of the

In view of the above three problems, let’s first ignore the technical implementation and assume what is the ideal solution to solve these three problems:

  • There is no texture cache, so we can add a texture memory cache in the application layer.
  • When the upper application layer already caches textures, the memory cache of the Native Bitmap can also be removed, leaving only the disk cache of image resources.
  • The memory cache of Flutter is only the texture cache and the ImageCache cache of Flutter. To avoid wasting memory resources, the two caches are combined into one

Therefore, the ideal solution is that there is only one memory cache in the whole App, which can cache both textures and Image data loaded by the Image Widget of Flutter.

The solution

ImageCache is officially provided and we can’t remove it, and there are places in the Xianyu App that use Image widgets. Now the solution is to cache the texture data in ImageCache as well. To use a texture, first fetch it from imageCache.

Let’s take a look at the existing Flutter image loading logic and how the images are cached

The ImageCache. PutIfAbsent method will be used to fetch the cache. The cache will use the existing loader method to construct the corresponding ImageStreamCompleter. The ImageStreamCompleter does the logic for loading the image.

When a buffer is hit, the putIfAbsent method returns the ImageStreamCompleter directly. The object holds the imageInfo. The ImageWidget renders the Image directly from the imageInfo UI.

Scheme 1: Extend ImageCache to cache textures

ImageCache provides only a putIfAbsent method for fetching the cache

At first we want to build the corresponding key, loader, and ImageStreamCompleter according to the method parameters, and then use the putIfAbsent method to fetch the cache as well.

When the image is successfully downloaded and decoded, the listener method is called back. In this method, the image is stored in the ImageCache cache queue

The listener callback takes two arguments, ImageInfo, which holds the Image data uI.image.

There is no way for the application layer to construct the UI.Image because this class is set to the application layer by the underlying Flutter engine after decoding the Image. There is no way for the application layer to actively set values. As a result, the value of imageSize cannot be computed in the listener and cannot be stored in the cache.

Solution 2: Customize ImageCache

Because the ImageCache cache queue is private, only the putIfAbsent method can store data in it. Then we only have another way, from the source of ImageCache, to customize ImageCache, and then extend its function.

Replace ImageCache with our custom #####

Because the ImageCache code provided by Flutter cannot be modified, we directly copy the source code of ImageCache, inherit ImageCache, and replace the PaintingBinding ImageCache with a custom one.

As shown in the figure: The PaintingBinding of Flutter exposes the createImageCache method. We inherit WidgetsFlutterBinding and rewrite this method to return our own ImageCache. You can also set different cache sizes for ImageCache.

Function extension for ImageCache #####

In order to modify the ImageCache code as little as possible, we define a new method to cache the texture directly and align the logic of the putIfAbsent method. The core code logic is as follows:

This method is implemented using the putIfAbsent logic. In order to cache the texture into ImageCache, the following key extensions are made:

  1. TextureCacheKey is a key that uniquely identifies a texture. This key is used to determine whether it is the same texture based on the width, height and URL.
  2. TextureImageStreamCompleter is texture management class, the class inheritance ImageStreamCompleter, internal hold texture data and download the success callback. When the cache is hit, the object is returned to the application layer and the Texture ID from it is rendered by the Texture Widget
  3. Invoked when missed cache incoming loader TextureImageStreamCompleter method construction, and will perform texture loading logic. At the same time will construct a listener callback, registered into TextureImageStreamCompleter.
  4. When the texture is loaded successfully, a callback is performed to the Listener method, which calculates the size of the texture, places it in the cache queue, checks if the cache size exceeds the maximum, and the oldest unused texture is discarded.

One important point to note here is that normal images are Dart objects that are automatically reclaimed by the Dart VM, but the actual data for our texture objects is in the Engine’s shared memory, so we need to manually manage the texture release. We reference the texture, and only when no widget holds the texture, When the reference count reaches zero, the release is true.

Similarly, the upper Texture Widget calls the interface provided by ImageCache when dispose is disposed to see if the Texture being used is cached or in use. The texture will only be released when it does not

The effect

We used the search results page as the test page, which had many baby images and various duplicate tabs. Use huawei Honor 20 to test the physical memory footprint before and after optimization.

The operation procedure is as follows: open the APP, enter the search result page, search the same keyword and enter the search result page. Then slide to browse 100 data after 10 seconds of silence, and finally stop the operation. During this period, physical memory is sampled once per second for 100s, and the following data is obtained

The blue curve is the memory occupied before optimization, and the orange curve is the memory occupied after optimization. It can be seen that the memory occupied is basically the same when entering. The drop in memory usage during sliding is caused by the GC starting to reclaim the App’s memory. Overall, the total memory footprint after optimization is lower than before optimization because GC also causes fewer burrs than before optimization.

Looking forward to

Although the above solution implements one memory cache within one App and stores textures and Flutter images, saving memory space and improving memory utilization, it still invades ImageCache source code. Subsequent upgrade and code maintenance of the Flutter Engine require additional work.

In addition, due to the loading of native images on the Flutter side, putIfAbsent is used for all images, and because the original image is loaded for all native images, there may be a situation in our app from time to time. One image may occupy several M of memory, so we directly add the big picture monitoring method to putIfAbsent. When the size of a loaded image exceeds 2 MB, data is reported, including the URL of the image, image usage information, and image size. In this way, we found several instances of Image misuse: loading the original Image directly using Image.network, or loading a large local resource using image.asset.