Author: Dong Hongping (Hiyuki), head of Didi Chuxing applet, head of MPX framework and core author
With the great success of small program in business, small program development in the domestic front-end field more and more attention, in order to facilitate the majority of developers better small program development, all kinds of small program framework also emerge in endlessly, showing a situation of flowers blossoming. But so far, the industry has not appeared a comprehensive, detailed, objective, fair small program framework evaluation report, for small program developers in the selection of technology to provide reference. This is the first in a series of articles I’ve been planning to publish that will provide a comprehensive, objective review of popular applet frameworks in the industry.
In this article, we test the runtime performance of the following frameworks (in no particular order):
- Wepy2 (https://github.com/Tencent/wepy) @ 2.0.0 – alpha. 20
- uniapp(https://github.com/dcloudio/u… @ 2.0.0-26120200226001
- MPX (https://github.com/didi/mpx) @ 2.5.3
- chameleon(https://github.com/didi/chame… @ 1.0.5
- mpvue(https://github.com/Meituan-Di… @ 2.0.6
- kbone(https://github.com/Tencent/kb… @ 0.8.3
- Taro next (https://github.com/NervJS/taro) @ 3.0.0 – alpha. 5
Among them, Kbone and Taro Next were tested with Vue as the business framework.
Runtime performance tests include the following dimensions:
- Frame runtime volume
- Page rendering time
- Page update time
- Local update time
- Number of setData calls
- SetData sends the data size
Framework performance test demo all stored in https://github.com/hiyuki/mp-… In, welcome developers to verify and correct errors and complete;
Test plan
In order to make the test results real and effective, I built two test scenarios based on common business scenarios, namely dynamic test scenario and static test scenario.
Dynamic Test Scenarios
In the dynamic test, the view is rendered dynamically based on data, and there are few static nodes. The view update time and setData call are the main test points in the test scene.
The dynamic test demo simulates the common long list + multi-tab scene in real business. There are two coupon list data in the demo, one is available coupon data and the other is not available coupon data. Only one of the data will be rendered and displayed in the view at the same time. You can simulate various operations on the list data and view display switches (cut tabs) in the upper action area.
Dynamic Test Demo
In the dynamic test, I externally used the function proxy to proxy the App, Page and Component constructor before initialization, and injected the setData intercepting logic in the Page onLoad and Component created hooks through the mixin. Monitor the setData calls of all pages and components, and count the view update time and setData calls of the applet. This test method can achieve zero intrusion of framework code, can trace to the small program setData behavior of the full amount and independent time-consuming calculation, strong universality, code implementation can view https://github.com/hiyuki/mp-…
Static test scenario
Static test simulates the scene of static pages in the business, such as pages of operation activities and articles. There are a large number of static nodes in the page, but there is no dynamic rendering of data. The initial ready time is the focus of test in this scene.
The static test demo uses HTML code from a technical article I published last year for a small program adaptation build that contains a lot of static nodes and text content.
Static Test Demo
Test process and data
The test data of all the following time consuming categories are obtained from the average value of 5 tests of the real machine in the WeChat applet, and the unit is ms. The test environment of iOS is iPhone 11, system version 13.3.1, WeChat version 7.0.12, and the test environment of Android is Xiaomi 9, system version Android10, and WeChat version 7.0.12.
In order to make the data presentation not too confusing and complicated, the data listed in the article are mainly the test results of IOS. The Android test results are consistent with that of IOS, and the overall time consumption is about 3-4 times higher than that of IOS. All the original test data are stored
https://github.com/hiyuki/mp-…
Since the introduction of Core-JS in Transform Runtime will affect the runtime size and runtime of the framework, and not all frameworks will turn on Transform Runtime at compile time, in order to align the test environment, The following tests were performed when Transform-Runtime was off.
Frame runtime volume
Since not all frameworks can use Webpack-Bundle-Analyzer to get an accurate package volume footprint, here I use the runtime volume of the framework by subtracting the volume of the demo project generated by each framework from the volume of the demo project written by native.
| DEMO Total Volume (KB) | Frame runtime volume (KB) | |
|---|---|---|
| native | 27 | 0 |
| wepy2 | 66 | 39 |
| uniapp | 114 | 87 |
| mpx | 78 | 51 |
| chameleon | 136 | 109 |
| mpvue | 103 | 76 |
| kbone | 395 | 368 |
| taro next | 183 | 156 |
The conclusion of this test is: Native > wepy2 > MPX > MPVue > Uniapp > Chameleon > Taro next > Kbone
Conclusion Analysis:
- WEPY2 and MPX have the best volume control when the frame is running;
- Taro Next and Kbone take up a lot of space due to their dynamic rendering nature, which generates recursive rendering templates/components in dist.
Page rendering time (dynamic testing)
We use the refresh page action to trigger the page reload. For most frameworks, the page rendering time is between the refresh action and the page executing onReady. However, for dynamic rendering frameworks like Kbone and Taro Next, the page executing onReady does not mean that the view is actually rendered. We set a special rule, within 1000ms triggered by onReady, when the setData callback occurs without any operation, the last triggered setData callback is used as the time to complete the page rendering to calculate the real page rendering time. The test results are as follows:
| Page rendering time | |
|---|---|
| native | 60.8 |
| wepy2 | 64 |
| uniapp | 56.4 |
| mpx | 52.6 |
| chameleon | 56.4 |
| mpvue | 117.8 |
| kbone | 98.6 |
| taro next | 89.6 |
The time spent in this test is not the same as the real rendering time. Since the applet itself does not provide Performance API, the real rendering time cannot be accurately calculated by JS test. However, from the data obtained, this data still has certain reference significance.
The conclusion of this test is: MPX ≈ chameleon ≈ uniapp ≈ native ≈ wepy2 > taro next ≈ kbone ≈ mpvue
Conclusion Analysis:
- Since MPVue renders all of its content on the page, Kbone and Taro Next use dynamic rendering technology, which takes a long time to render the page. The rest of the frameworks are not much different.
Page update time (no background data)
Here, the background data is defined as the data that exists in the data but is not used in the current page rendering. In this demo scene, it is the data of unavailable coupons. At present, under the condition that unavailable coupons are 0, various operations will be carried out on the list of available coupons and the updating time will be counted.
The update time is calculated from the time the data operation event is triggered to the completion of the corresponding setData callback
In MPVUE, the current timestamp (new Date) is used as the timeout and the setData is throttled with a timeout of 50ms. This mode has serious problems. When the execution time of a single render synchronization process in VUE exceeds 50ms, the setData triggered by the subsequent component patch will break through the throttling limit. Invalid high-frequency calls to setData ata frequency of 50ms each. In the performance test demo, the interface completely freezes when the number of coupons exceeds 500. In order to run the whole test process smoothly, I made a simple fix to this problem. I used setTimeout to rewrite the part of the Throttling to ensure that setData would be called to send the merged data after the synchronous execution of the single rendering process of Vue. After that, all performance tests of MPVue were conducted based on this patch version. The patch version is stored
https://github.com/hiyuki/mp-…
Theoretically speaking, the performance of native must be the ceiling of all frameworks under the premise of optimization. However, in daily business development, we may not be able to optimize every setData. In the following performance tests, all native data are sent in the form of full amount after modification.
In the first test, we incrementing the number of available coupons from 0 to 1000 using the new available coupons (100) operation:
| 100 | 200 | 300 | 400 | 500 | 600 | 700 | 800 | 900 | 1000 | |
|---|---|---|---|---|---|---|---|---|---|---|
| native | 84.6 | 69.8 | 71.6 | 75 | 77.2 | 78.8 | 82.8 | 93.2 | 93.4 | 105.4 |
| wepy2 | 118.4 | 168.6 | 204.6 | 246.4 | 288.6 | 347.8 | 389.2 | 434.2 | 496 | 539 |
| uniapp | 121.2 | 100 | 96 | 98.2 | 97.8 | 99.6 | 104 | 102.4 | 109.4 | 107.6 |
| mpx | 110.4 | 87.2 | 82.2 | 83 | 80.6 | 79.6 | 86.6 | 90.6 | 89.2 | 96.4 |
| chameleon | 116.8 | 115.4 | 117 | 119.6 | 122 | 125.2 | 133.8 | 133.2 | 144.8 | 145.6 |
| mpvue | 112.8 | 121.2 | 140 | 169 | 198.8 | 234.2 | 278.8 | 318.4 | 361.4 | 408.2 |
| kbone | 556.4 | 762.4 | 991.6 | 1220.6 | 1468.8 | 1689.6 | 1933.2 | 2150.4 | 2389 | 2620.6 |
| taro next | 470 | 604.6 | 759.6 | 902.4 | 1056.2 | 1228 | 1393.4 | 1536.2 | 1707.8 | 1867.2 |
Then we click in order to delete the available coupons (All) > add the available coupons (1000) > update the available coupons (1) > update the available coupons (All) > delete the available coupons (1) :
| delete(all) | add(1000) | update(1) | update(all) | delete(1) | |
|---|---|---|---|---|---|
| native | 32.8 | 295.6 | 92.2 | 92.2 | 83 |
| wepy2 | 56.8 | 726.4 | 49.2 | 535 | 530.8 |
| uniapp | 43.6 | 584.4 | 54.8 | 144.8 | 131.2 |
| mpx | 41.8 | 489.6 | 52.6 | 169.4 | 165.6 |
| chameleon | 39 | 765.6 | 95.6 | 237.8 | 144.8 |
| mpvue | 103.6 | 669.4 | 404.4 | 414.8 | 433.6 |
| kbone | 120.2 | 4978 | 2356.4 | 2419.4 | 2357 |
| taro next | 126.6 | 3930.6 | 1607.8 | 1788.6 | 2318.2 |
At the beginning of this test, my Update (All) logic was to loop through each list item to update, like this
listData.forEach((item)=>{item.count++}), it is found that the execution interface will be completely stuck in Chameleon framework. Tracking found that there is no asynchronous merge processing of setData in Chameleon framework, but it is sent directly synchronously when the data changes. In this way, when the data volume is 1000, updates in this way will trigger the setData for 1000 times at high frequency. Resulting in interface jamming; To this end, in the test demo of the Chameleon framework, I adjusted the logic of UPDATE (ALL) to generate an updated ListData of deep clone, and then assigned its whole value to this.listData, so as to ensure that the test could work normally.
The conclusion of this test is: Native > MPX ≈ Uniapp > Chameleon > mpvue > wepy2 > taro next > Kbone
Conclusion Analysis:
- MPX and Uniapp have carried out perfect diff optimization within the framework. With the increase of data volume, the new time of the two frameworks did not increase significantly.
- Wepy2 sets the props data as well when the data changes, which causes a large amount of ineffective performance loss in this scenario, resulting in poor performance.
- Kbone and Taro Next use a dynamic rendering solution that sends a lot of data describing the DOM structure with each new update. At the same time, dynamic recursive rendering takes much longer than regular static template rendering, making both frameworks much longer than other frameworks in all update scenarios.
Page update time (with background data)
After refreshing the page, we created background data with the newly added unusable voucher (1000), observed whether setData would be triggered by this operation and counted the time
| back add(1000) | |
|---|---|
| native | 45.2 |
| wepy2 | 174.6 |
| uniapp | 89.4 |
| mpx | 0 |
| chameleon | 142.6 |
| mpvue | 134 |
| kbone | 0 |
| taro next | 0 |
MPX setData optimization inspired by Vue, using the rendering function generated at compile time to track the template data dependency, in the background data changes will not make setData calls, while Kbone and Taro Next used dynamic rendering technology to simulate the web underlying environment, The same effect was achieved by running the VUE framework fully on top.
Then we performed the same operation as before without background data, first incrementing by 100:
| 100 | 200 | 300 | 400 | 500 | 600 | 700 | 800 | 900 | 1000 | |
|---|---|---|---|---|---|---|---|---|---|---|
| native | 88 | 69.8 | 71.2 | 80.8 | 79.4 | 84.4 | 89.8 | 93.2 | 99.6 | 108 |
| wepy2 | 121 | 173.4 | 213.6 | 250 | 298 | 345.6 | 383 | 434.8 | 476.8 | 535.6 |
| uniapp | 135.4 | 112.4 | 110.6 | 106.4 | 109.6 | 107.2 | 114.4 | 116 | 118.8 | 117.4 |
| mpx | 112.6 | 86.2 | 84.6 | 86.8 | 90 | 87.2 | 91.2 | 88.8 | 92.4 | 93.4 |
| chameleon | 178.4 | 178.2 | 186.4 | 184.6 | 192.6 | 203.8 | 210 | 217.6 | 232.6 | 236.8 |
| mpvue | 139 | 151 | 173.4 | 194 | 231.4 | 258.8 | 303.4 | 340.4 | 384.6 | 429.4 |
| kbone | 559.8 | 746.6 | 980.6 | 1226.8 | 1450.6 | 1705.4 | 1927.2 | 2154.8 | 2367.8 | 2617 |
| taro next | 482.6 | 626.2 | 755 | 909.6 | 1085 | 1233.2 | 1384 | 1568.6 | 1740.6 | 1883.8 |
Then click Statistics item by item according to the following table operation order
| delete(all) | add(1000) | update(1) | update(all) | delete(1) | |
|---|---|---|---|---|---|
| native | 43.4 | 299.8 | 89.2 | 89 | 87.2 |
| wepy2 | 43.2 | 762.4 | 50 | 533 | 522.4 |
| uniapp | 57.8 | 589.8 | 62.6 | 160.6 | 154.4 |
| mpx | 45.8 | 490.8 | 52.8 | 167 | 166 |
| chameleon | 93.8 | 837 | 184.6 | 318 | 220.8 |
| mpvue | 124.8 | 696.2 | 423.4 | 419 | 430.6 |
| kbone | 121.4 | 4978.2 | 2331.2 | 2448.4 | 2348 |
| taro next | 129.8 | 3947.2 | 1610.4 | 1813.8 | 2290.2 |
The conclusion of this test is: Native > MPX > Uniapp > Chameleon > MPvue > wepy2 > Taro next > Kbone
Conclusion Analysis:
- The three frameworks with template data tracking capabilities, MPX, Kbone, and Taro Next, did not significantly increase their time in scenarios with background data.
- The diff in WEPY2 is not accurate enough and the time is not significantly changed.
- For the rest of the frameworks, the deep diff of the background data is carried out for each update, which increases the time consumption to a certain extent.
Page update time (large data volume scenario)
Since the rendering of MPVue and Taro Next is all carried out in the page, while the rendering scheme of Kbone will add a large number of additional custom components, these three frameworks will all collapse to the white screen when the number of coupons reaches 2000. We excluded these three frameworks from the test of page update time for the rest frameworks in scenarios with large amounts of data
First, increase the number of available coupons to 5000 by using new available coupons (1000) in the scenario without background data:
| 1000 | 2000 | 3000 | 4000 | 5000 | |
|---|---|---|---|---|---|
| native | 332.6 | 350 | 412.6 | 498.2 | 569.4 |
| wepy2 | 970.2 | 1531.4 | 2015.2 | 2890.6 | 3364.2 |
| uniapp | 655.2 | 593.4 | 655 | 675.6 | 718.8 |
| mpx | 532.2 | 496 | 548.6 | 564 | 601.8 |
| chameleon | 805.4 | 839.6 | 952.8 | 1086.6 | 1291.8 |
Then click Add Unusable Coupon (5000) to increase the amount of background data to 5000, and then test the time to increase the number of available coupons to 5000:
| back add(5000) | |
|---|---|
| native | 117.4 |
| wepy2 | 511.6 |
| uniapp | 285 |
| mpx | 0 |
| chameleon | 824 |
| 1000 | 2000 | 3000 | 4000 | 5000 | |
|---|---|---|---|---|---|
| native | 349.8 | 348.4 | 430.4 | 497 | 594.8 |
| wepy2 | 1128 | 1872 | 2470.4 | 3263.4 | 4075.8 |
| uniapp | 715 | 666.8 | 709.2 | 755.6 | 810.2 |
| mpx | 538.8 | 501.8 | 562.6 | 573.6 | 595.2 |
| chameleon | 1509.2 | 1672.4 | 1951.8 | 2232.4 | 2586.2 |
The conclusion of this test is: Native > MPX > Uniapp > Chameleon > WEPY2
Conclusion Analysis:
- In large data volume scenarios, the underlying performance differences between the frameworks become more obvious, with MPX and Uniapp still maintaining near-native good performance, while Chameleon and WEPY2 experienced significant performance degradation.
Local update time
When the number of available coupons is 1000, we click on any one of the available coupons to trigger the selection state, so as to test the local update performance
| toggleSelect(ms) | |
|---|---|
| native | 2 |
| wepy2 | 2.6 |
| uniapp | 2.8 |
| mpx | 2.2 |
| chameleon | 2 |
| mpvue | 289.6 |
| kbone | 2440.8 |
| taro next | 1975 |
The conclusion of this test is: Native ≈ Chameleon ≈ MPX ≈ Wepy2 ≈ Uniapp > MPVue > Taro Next > Kbone
Conclusion Analysis:
- It can be seen that all the local update time of the framework using native custom components for componentization implementation is very low, which is enough to prove the excellence and importance of native custom components of small programs.
- MPVUE uses page updates, and local update time increases significantly.
- Kbone and Taro Next have high performance overhead for recursive dynamic rendering, resulting in an equally high local update time.
SetData call
We set the count and size options of ProxySetData to true to enable the number and volume statistics of SetData. After the reconstruction, we performed a series of operations according to the following procedure, and counted the number of calls to SetData and the volume of data sent.
The operation process is as follows:
- 100 progressively increasing available coupons (0->500)
- Switch to Unusable Coupons
- New Unusable Coupon (1000)
- 100 progressively increasing available coupons (500->1000)
- Update available vouchers (ALL)
- Switch to available coupons
After the completion of the operation, we used getCount and getSize methods to obtain the cumulative number of setData calls and data volume, in which the data volume was calculated as JSON. Stringify was followed by UTF-8 encoding to calculate the volume, and the statistical results were as follows:
| count | size(KB) | |
|---|---|---|
| native | 14 | 803 |
| wepy2 | 3514 | 1124 |
| mpvue | 16 | 2127 |
| uniapp | 14 | 274 |
| mpx | 8 | 261 |
| chameleon | 2515 | 319 |
| kbone | 22 | 10572 |
| taro next | 9 | 2321 |
MPX > Uniapp > Native > Chameleon > Wepy2 > Taro Next > MPVUE > KBONE MPX > Uniapp > Native > Chameleon > Wepy2 > Taro Next > MPVUE > KBONE
Conclusion Analysis:
- MPX framework successfully realizes the optimization of theoretical setData;
- Uniapp was next due to its lack of template tracking capability;
- Chameleon generates a lot of invalid setData calls because the component makes an unnecessary setData call every time it is created, but the data itself is sent through diff, which does a good job in the amount of data sent;
- The components of WEPY2 will call setData to send the updated props data when the data is updated. As a result, a large number of invalid calls are generated, and the diff accuracy is insufficient, and the amount of data sent is large.
- Taro Next controls the number of data sent to 9 times because the upper layer is completely based on VUE. However, due to the need to send a large amount of DOM description information, the amount of data sent is large.
- Because MPVUE uses a long data path to describe the components corresponding to the data, it also produces a large amount of data transmission.
- Kbone’s call to setData is not very well controlled. It still sends data 22 times while running Vue on top, and sends a huge amount of data, up to a staggering 10MB in this process.
Page rendering time (static test)
The page rendering time here is the same as in the dynamic test scenario described earlier, and the results are as follows:
| Page rendering time | |
|---|---|
| native | 70.4 |
| wepy2 | 86.6 |
| mpvue | 115.2 |
| uniapp | 69.6 |
| mpx | 66.6 |
| chameleon | 65 |
| kbone | 144.2 |
| taro next | 119.8 |
The conclusion of this test is: Chameleon ≈ MPX ≈ Uniapp ≈ Native > wepy2 > MPVue ≈ Taro next > Kbone
Conclusion Analysis:
- With the exception of Kbone and Taro Next, which take longer to render dynamically, and MPVue, which uses page templates to render poorly, static page rendering is almost as good as native rendering.
conclusion
Combined with the above test data, we get the final run-time performance ranking of the small program framework: MPX > Uniapp > Chameleon > wepy2 > MPVUE > Taro Next > Kbone
A little bootleg
The dynamic rendering technology used by Kbone and Taro Next is not as good in terms of performance, but I still think it’s a great technical solution. Although performance tests and comparisons have been carried out from the beginning to the end in this paper, performance is not the whole framework. Development efficiency and high availability are still the focus of the framework. Development efficiency is believed to be the original intention of all framework design, but high availability has been largely ignored. From this point of view, Kbone and Taro Next are very successful. Unlike the translation ideas of the past, this approach of smoothing out the underlying rendering environment enables the full operation of the upper web framework, which brings a great improvement in the usability of the framework. It is very suitable for the migration and development of simple small programs for operations.
I dominate the development framework of MPX https://github.com/didi/mpx chose another way to solve the usability problem, which is based on the small program to enhance its capability of native grammar, this can avoid translation web framework of uncertainty and instability, At the same time can also bring very close to native performance, for complex business small program developers, very recommended to use. In output across, MPX can improve all the small application platform and the web platform supporting the industry isomorphism output, drops inside is the most important the most complex small program – drops travel small program based on MPX development completely, and by using the framework across the ability WeChat and pay treasure to synchronize business entrance iteration, greatly improve the efficiency of business development.