Writing in the front

In order to solve the problem of small program generating shared photos, we started the Artist Project, a small program image generation library. The project is open source and can be moved to github.com/Kujiale-Mob… . One of the drawbacks of Canvas drawing is its drawImage method, which can be directly set to the URL of the web image in the IDE, but cannot be done on the real machine. After this hole, we need to download the image to the local, before drawing. So in your small program, if there are frequent drawing some pictures of the needs, and need to use the network picture material, which leads to each drawing to re-download the material picture, produce a big drawing performance problems.

The applet itself does not provide a caching mechanism such as file LRU. In order to make our artist plan image generation faster, we developed a small program file for LRU storage related code. In this way, we do not need to repeatedly download the drawing material that may be used frequently, which greatly increases the drawing speed.

Introduce the small program cache system

Small program cache is divided into data cache, and file cache two parts. File caching is divided into temporary file caching and local file storage. The limit for storing local files is 10 MB.

Data cache

We can use small programs to provide a set of asynchronous and synchronous methods to add, delete and check structured data. The storage limit of the same mini program for the same wechat user is 10MB. If you run out of space you will LRU at the applet level, that is, the data cache area of the infrequently used applet will be completely cleared.

See wechat official document for details:

Developers.weixin.qq.com/miniprogram…

Note: Data caches are shared in the experience, development, and online editions, and are not isolated.

Temporary file cache

DownloadFile or wx.chooseimage, we will get a temporary path to the file or image. The document says that the life cycle of the temporary path is during the startup of the applet.

However, there is no limit on the size of the file, so in theory you can temporarily cache any file of any size, although if it is too large it will cause some magic error.

The local store

After obtaining the temporary file, we can call the interface wx.savefile to store the temporary file in the local space, which is limited to 10M. If the storage is full, subsequent files will not be able to be stored successfully and an error will be reported indicating that the maximum storage limit has been exceeded.

What we need to do now is to open up a space on this local storage space as the storage space for our downloaded files. Due to the limited space, we need to carry out LRU management on this space.

See the following documentation for interfaces related to local storage:

Developers.weixin.qq.com/miniprogram…

Note: The temporary file path is invalid after successfully calling saveFile. Remember to remember.

File LRU storage implementation

The local storage of small program side has a limit of 10M, but there is no LRU. Now we need to combine the three storage methods of small program mentioned above to achieve a set of LRU mechanism for small program file download.

Data structure design

{
'key': {
        'path': // The file storage path'time': // The timestamp is used to record the last access time of the file. When the storage is insufficient, the furthest unaccessed file will be selected for deletion'size': // File size}....'totalSize': // Total current size of all stored files}Copy the code

Where we use the downloaded URL as the key. The above data structures will exist in the data cache (which we will later call the storage) and will be read into memory from the storage when the downloader is built. Later file operations are synchronized to the file information recorded in the storage in real time.

You can think of the path as a pointer to the actual file where the basic file information is stored.

Overall process design

Fault tolerance

Because storage and file operations are asynchronous, it is possible that they are inconsistent. There are two kinds of inconsistencies here

In the first case, information about a file in storage is deleted, but the file itself is not deleted because of a magic error. In addition, the file is added successfully, but the storage is not added successfully also belongs to this case.

Second, the file information in the storage fails to be deleted, but the file is deleted.

The above two properties are different, so they need to be treated differently. For the first type, the file storage space is inconsistent with the file information recorded in the storage, that is, there are stray files (not tracked by the storage).

The second, which is equivalent to the presence of null Pointers, is absolutely necessary to avoid, as it can cause you to use a file that does not exist. Can directly lead to serious bugs.

In view of the above two special cases, the following fault-tolerant processing has been done. First of all, we must ensure that the file deletion operation must be successful after storage. This ensures that the second one will not go wrong.

And for the first case of the free file. We will take advantage of the timing of saveFile. If you have a stray file, you end up with an inconsistent calculation of the total size of your space, which may end up with a situation where your external logic thinks you can store it, but the actual storage space is full, and you end up with a saveFile error, and after saveFile fails, for whatever reason, Let’s all clear out the contents related to this policy store and start again. Since we always have tempFilePath in our pocket, even if this happens, it won’t affect the user. It just affects the user experience a little bit (after all, there is no previous cache).

Note: The reason I don’t want to guarantee the first case in the same way as the second case is because I don’t feel the need to waste performance and impact the user experience by dealing with rare error scenarios. As long as we take care of it, the system will still work, even if such a mistake happens.

Write in the back

Applets have lots of potholes. At present, many small programs on the market performance experience is not very good. So in order to do a high-performance small program, we need to spend a lot of time to trial and error, pondering. If you step on a pit, you will die.

At present, this LRU file storage mechanism has been used in our open source Painter library. If you are interested, please go to github.com/Kujiale-Mob… .