Author: sky – ADMIN

An overview of the

In the second half of 2019, in order to transfer didi Chuxing of Wechat Wallet/Alipay Jiugongge entry into a mini program, the team carried out a large number of function upgrades and completion of the mini program. A series of problems and challenges were also encountered and overcome in the whole process, especially the package volume problem. Next, I will introduce the efforts and precipitation of Didi Chuxing mini program in volume control.

background

Wechat’s requirement for the volume of small program package is that the total volume shall not exceed 12M, and the volume of main package and single subpackage shall not exceed 2M. Although alipay’s calculation method for the volume of small program package is slightly different from wechat’s, it is almost the same overall.

From the beginning of 2018 to the beginning of 2013, didi Chuxing’s mini program only carried online car-hailing services with few business demands, which could be handled within the main package. In the second half of the year, in order to transfer the entrance of wechat wallet/Alipay to mini program, mini program began to add many business lines, such as public transportation/designated driving/car service/bike/hitchhiking. Meanwhile, the business demand of online car booking also needs to be made up comprehensively, and the business volume and code volume have exploded together.

Didi Chuxing has diversified travel businesses, including express, private car, taxi, luxury car, carpooling, bicycle, proxy driving, hitch car, bus, car life and many other business lines. The most important and frequently used pages of the whole Didi Chuxing mini program are the home page and the order details page. The home page carries the demand expression of each business line, and the order details page of each business line carries the specific travel order display logic. In addition, there are various functional pages such as personal center, marketing page, Settings, history itinerary.

According to the product logic of Didi Chuxing, the demand expression logic of all lines of business is carried on the home page. In order to have a good switching experience, a single top page is adopted to display lines of business on the home page. That is, each line of business provides a requirement expression component on the home page. When the user switches the top leading line of business, the corresponding line of business component will be switched out.

In this design, all the requirement expression logic of the business line is concentrated in the single page of the home page. As a result, in the process of business iteration, the volume of the main package carrying the home page increases rapidly, and it soon hits the maximum of 2M per package of the small program platform, bringing great obstacles to the subsequent business iteration and development. Therefore, the control of package volume is a big problem we face in the process of small program development.

The volume control

Next, we will introduce a series of optimization control practices on the volume of small program packages in the iterative development process of Didi Chuxing small program.

Basic optimization means

For small programs, the basic package volume optimization means include: resource compression/code redundancy removal/resource CDN/asynchronous loading

In Web development, WebPack provides a wide range of code optimization capabilities, including dependency analysis, module de-duplication, code compression, tree shaking, Side Effects, etc. These capabilities make it easy to compress resources and remove code redundancy. Didi Chuxing mini program is developed based on The open source mini program framework Mpx(github.com/didi/mpx) of Didi chuxing. The compilation and construction of Mpx framework is completely based on WebPack, which is compatible with the internal ecology of WebPack. Therefore, the package volume can be optimized by using the above capabilities.

In the small program, some static resources (such as images and videos) can be loaded by CDN address. We will try our best to compress relevant resources and put them on CDN to avoid the occupation of package volume by these resources.

In the scenario of small program, asynchronous loading cannot be easily carried out through script tags as in the Web, but in the later stage of small program platforms, subcontracting loading schemes are successively supported to achieve this capability. Since subcontracting loading is a special technical specification of small program, Webpack cannot directly support it. Therefore, the Mpx framework is well adapted to support this technical specification, and we will discuss the application of this capability in more detail later.

In addition, Mpx framework also carries out a lot of package volume optimization adaptation for small program scenarios, such as minimizing the footprint of package volume during framework runtime (56Kb after compression), packaging and building referenced pages/components on demand, declaration of common styles for style reuse, and extraction of common modules in subcontracting, etc.

With these capabilities of the Mpx framework, it is possible to build a small, preliminarily optimized package with little additional configuration.

There is also a similar option for “automatic compression confusion when uploading code” in the wechat developer tool, but the volume calculated when uploading code in the developer tool is directly calculated the volume of the current project code, not according to the volume after compression. Therefore, if you are developing with native applets, your source code will most likely be further compressed to save space.

Analysis of the volume

Although the framework has provided many optimizations in terms of volume control, as the business iterated we found that the main package was still too large.

When we have a big main package, we need to figure out, what’s in the main package? Why are they so big?

Students who are developing using native applets or other non-WebPack-based frameworks may encounter this problem by simply looking at the file size on their hard disk. As a result, the size ratio of each module may not be intuitive. A webpack plugin called Webpack-bundle-Analyzer can be used to assist analysis.

For example, if this is a demo written using the Mpx framework, the NPM run build –report interface will look like this:

Note that the demo consists of third-party libraries such as Moment/Lodash/socket-retry/core-js. The size and interdependencies of the individual libraries can also be clearly seen.

A similar picture can be seen for didi Chuxing applet, which codes are composed of the whole project.

In addition, the front-end development of Didi Chuxing has always adopted “source code compilation”, which enables only one common dependency in the whole project to be shared. In short, it also helps reduce project code volume. Related: https://github.com/DDFE/DDFE-blog/issues/23

To perfect play the effect of the source code to compile, need to build together to establish a complete set of source of upstream and downstream of the ecology, such as additional qualifications to rely on party in public announcement rely on, you should use peerDep or devDep to declare some public reliance on, the mutual dependence should be unified statement in the main project, avoid the version put on two different public dependence, That will increase the volume. Due to the fact that Didi Chuxing mini program involves many business lines and teams, some teams may not know about this, so the above deterioration scenario may actually occur in the code. The diagrams make it easy to spot such problems and motivate the teams involved to clean up these duplicate dependencies.

At the same time, according to the volume analysis chart, we focus on the analysis of the large files, a round of business code sorting and simplification, delete some useless code, simplify the websocket message body description file, etc.

Configuration of the subcontract

Subcontracting is a scheme similar to web asynchronous introduction given by small programs. Some pages that are not needed in the initial entry can be put into the subcontracting, and the subcontracting can be downloaded when jumping to the corresponding page. Putting these pages and their affiliated resources into the subcontracting can effectively reduce the volume of the main package.

Mpx framework provided preliminary support for the subcontracting specification in the early stage, and the resource access rules were consistent with wechat, mainly judging whether to output to the main package or subcontract according to the directory of resources stored. After we had this ability, we pulled out the schedule page to subcontract, and about 200 K space was extracted.

With the successful split of the travel page, we started to subcontract all the non-home page code, such as start/end selection and personal center. As well as the access of some business lines through NPM, we also tried to subcontract all the non-home page codes of these business lines.

As an aside, THANKS to MPX’s early design of packages for business portfolio solutions, businesses can be easily developed independently and can be extremely easily integrated. After that, we find that the json configuration design of wechat subcontracting is similar to packages, so we support wechat subcontracting on this basis. Users only need to add a query mark to the subcontracting name on the basis of the original packages.

After the removal of various subcontracts, the overall project structure is roughly shown as follows:

After the initial subcontracting work was completed, a total of nearly 400K space was removed from the main package to the subcontracting.

Fine management of subcontracting resources

As mentioned above, the subcontracting processing rule of Mpx framework in the early stage was to collect the resources under the subcontracting path into the subcontracting according to the way of wechat. Resources managed by NPM are collected into the main package because they are in node_modules and do not belong to any subcontracting path.

For example, before we had the trip page subcontracting, the trip page has its own state management store under the path of the trip page subcontracting, which will be collected into the trip page subcontracting. The trip page also uses the wrapped didi-Socket library, which is a common NPM package, even though it is only used in the trip page subpackage, because its path is under node_modules, it is collected into the main package.

In the early design, the resources and the home page of the schedule page were separated and stored independently in their respective paths. The schedule page was the main part of the subcontracting process of the first phase, which just matched the characteristics of the subcontracting process in the early stage of Mpx, so it could be better collected and subcontracted into the schedule page.

As the business iterates, and a large number of subsequent lines of business are accessed through NPM, there will be a large number of NPM package resources, which are all in node_modules, and therefore all collected into the main package.

So the Mpx framework underwent a series of transformations:

  1. During the dependency collection process of the build, we mark the collected dependencies and record which subcontracting it was introduced by. Once it has only one subcontract introduced, it will be exported to that subcontract.
  2. We will automatically generate each subcontracted cacheGroups in SplitChunksPlugin based on the user-defined subcontracted configuration, extracting the subcontracted reuse modules into the subcontracted bundles.
  3. For components and static resources, if they are referenced by multiple subpackages but not in the main package, to ensure the optimal volume of the main package, these resources will produce multiple copies of output to the corresponding subpackages, without occupying the main package volume.

In this way, no matter where the resource referenced in the subcontract was originally located, it will be output to the subcontract directory of Dist as far as possible in the final output to avoid occupying the space of the main package.

After this change, the project structure looks the same as before, but thanks to the upgrade of Mpx’s ability to handle the subcontracting resources, we were able to successfully export the NPM resources referenced in the line of business subcontracting to their subcontracting directory.

Cover scheme

Later, Didi Chuxing mini program needed to replace the original WebApp entry in wechat/Alipay, and the business lines accessed by the mini program increased rapidly and the package volume increased rapidly.

As mentioned earlier, the main reason for the growth of this section is that all lines of business are displayed on the main page. This is also determined by the business characteristics. Didi Chuxing provides a rich travel product line, including express, private car, taxi, luxury car, carpool, bicycle, designated driving, hitch, and other products, which users may need to repeatedly switch between and choose. This process also needs to retain information such as the starting and ending models, and must be a page switching component with a very complex set of large state management to be relatively smooth implementation. Unlike some e-commerce/information platforms, different functions can not be split into different pages, so that users can enter the sub-page through the menu of the home page and then operate. The home page only bears the entrance and only has less business logic, so it will be much easier to subcontract.

Therefore, each line of business should provide home page components for access. This component will be used on the home page, so it can’t be pulled out of the package anyway. Finally, the volume of the main package section of the entire home page can be divided into two parts: the base library and the business code. The volume ratio between the two is about 1M for the common dependency base and 1M for the business code.

The huge volume of the base library is mainly due to the large number of business lines and business teams of Didi Chuxing, and each party has its own basic dependence. For example, the long link communication PB data description file that online ride-hailing relies on, the large number computing library that map will depend on, the CML framework runtime that ride-hailing relies on, the communication gateway library that proxy driving relies on, and the common component library and Polyfill, etc.

Therefore, the problems faced by Didi Chuxing mini program could not be solved quickly in a short time with pure technical solutions, so we made an engineering structure adjustment, which can be called cover page solution, to solve the main package problem.

To put it simply, the cover plan is to make a cover with the Didi Chuxing Logo as the startup page. Once the page is loaded, it will immediately jump to another page, which actually bears the business and is subcontracted.

The significance of this operation is that the main package is left with only the infrastructure/libraries that all parties depend on, and the business is all taken away into subcontracting.

This is the structure diagram of the project after the completion of the cover scheme. Before, a large chunk of the home page business logic was extracted into the home page subcontracting.

The result of such a moving operation is that we can have a main package space of 2M to accommodate the basic public library, and a subcontract of 2M to accommodate the previously mentioned “big home page” of Didi Chuxing integrating various businesses. At that time, there were almost 1.2m main packages and 800K+ business main and subcontracting.

The most outstanding point of this transformation is that the volume growth generated by the subsequent business iterations is almost all in the main and subcontracting of the business, and the remaining 1.1m + space is relatively sufficient for the business iterations. Under ideal conditions, the volume of the main package can remain unchanged for a long time, so that the volume of the main package will not exceed the standard due to the continuous development of business requirements, and there is no need to worry about the volume of the main package.

Of course, as you can see, the scheme itself is not reducing any volume, just a change of position. In addition, there are some drawbacks to the cover page solution, such as a slower display of the first screen business due to more content to load, but the applets themselves have a better ability to cache resources, so it is acceptable.

Compared to the volume problem of stuck requirements iteration and product line access, the current solution can at least solve the problem of whether or not. Our development team will continue to pay attention to the volume issue in the future, to see if there will be any product solution changes or the mini program itself provides some solutions to further optimize this part.

conclusion

The Mpx framework has done a lot of work on package volume control, and has very good support for small program subcontracting in NPM scenarios.

On the basis of framework support, the Didi Chuxing mini-program team has managed to control the volume of the huge and complex Didi Chuxing mini-program package within the limits of the platform by organizing business dependence, making full use of subcontracts, adjusting interactive programs and a series of other means without hindering business development.

Hope this article can bring some inspiration to the small program developers who encounter problems in package size, welcome to leave a message.