Brief introduction:Performance optimization is a systematic and integrated matter, imprinted in the various details of the project development link, but also reflects the depth of technology in the big battlefield. Based on the complex system of Quick BI, this paper introduces the ideas and means of performance optimization in detail, as well as the thinking of systematization.
– For more information about digital intelligence transformation and data center platform, please join Aliyun Data center platform exchange group — digital intelligence club and follow the official WeChat official general account (scan QR code or click here to join).
China’s official website https://dp.alibaba.com/index ali cloud data
Performance has always been an unavoidable topic at the technical level, especially in medium to large complex projects. Just like the performance of a car, comfort and practicality should be ensured while pursuing top speed. Each link of car manufacturing, parts integration, engine tuning and so on will ultimately affect user sensation and business achievement, as shown in the following figure, the impact of performance on revenue.
Performance optimization is a systematic and integrated matter, imprinted in the various details of the project development link, but also reflects the depth of technology in the big battlefield. Next, I will take Quick BI’s complex system as the background, dive into the whole idea and means of performance optimization, as well as the systematic thinking.
How do you locate performance problems?
Generally speaking, we are sensitive to the frame rate of the animation (within 16ms), but if there is a performance problem, our actual motion sense may be just one word: “slow”, but this does not help us to solve the problem, so we need to analyze the whole link behind the word.
The above diagram shows the general processing flow for browsers. Based on our scenario, I have abstracted it into the following steps:
As can be seen, there are two main time consuming stages:
Stage 1: Download Resource Pack
Script Execution & Fetch Data
How to go deep into these two stages, we will generally use the following main tools to analyze:
Network
The first tool we’ll use is Chrome’s Network, which can help us initially identify bottlenecks:
As shown in the example shown in the figure, the entire page can be seen at a glance in the Network: load time (Finish), load resource size, number of requests, time and time point of each request, resource priority, etc. This is obvious from the example above: the entire page loads a large resource, approaching 30MB.
Coverage of code
For complex front-end projects, the products of the project construction are usually redundant or even unused. These invalid loaded codes can be analyzed in real time through the Coverage tool:
As you can see from the example above: 19.5MB of the entire page is unused (executed) of 28.3MB, and the engine-style.css file is used less than 0.7% of the time
Resources in a larger version
We already know that front-end resource utilization is very low, so what is the invalid code being introduced? In this case, we need to use Webpack-Bendle-Analyzer to analyze the entire build artifact (Stats can be output via webpack –profile –json=stats.json) :
As an example above, we can see the problem with build artifacts in conjunction with our current business:
First, the initial package is too large (common.js)
Second, there are multiple duplicate packages (MomentJS, etc.)
Third, dependent third-party packages are too large
Module dependency
Have the resources to build a larger version, we can also know about the optimization of the points, but in a system, hundreds of modules are generally organization by means of reference each other together, packaging tools through dependencies to build it together (such as into common. Js a single file), want to remove a module code directly or may not be easy, This may require a certain degree of unbundling, using tools to clarify the dependencies of the modules in the system, and optimizing by adjusting the dependencies or loading methods:
In the image above we are using the official Webpack Analyse tool (other tools include Webpack-xray, Madge), just upload the resource diagram stats.json to get the overall dependency diagram
Performance
Chrome provides a very powerful tool for analyzing what to use in the “Execute & Fetch” section.
As shown in the above example, we can find at least a few points: main process serialization, long tasks, and high frequency tasks.
How to optimize performance?
Combined with the analysis tools mentioned just now, we have basically covered the two major stages of “resource pack download” and “execution & number extraction” mentioned just now, and the fundamental problem solution has been gradually figured out in the continuous analysis. Here I will give some good optimization ideas and effects based on the scene we have here
Large packages are loaded on demand
You should remember that front-end projects (such as Webpack) usually start from entries and find a dependency tree (direct dependency) to produce multiple JS and CSS bundles or trunk from this tree. Once a module appears in the dependency tree, when the page loads an entry, The module is also loaded.
Therefore, our idea is to break this direct dependency and use asynchronous dependency for the modules at the end, as follows:
Change the synchronous import {Marker} from ‘@antv/l7’ to asynchronous so that the dependent Marker forms a chunk at build time and the Thunk is loaded only when this code is executed (as needed), thereby reducing the size of the first-screen package.
However, there is a problem with the above scheme. The build will treat the entire @AnTV/L7 as a chunk instead of Marker part code, resulting in the invalidability of the TreeShaking of this chunk and a large volume. We can solve this using build sharding:
As above, firstly create the sharding file of Marker to make it have the ability of TreeShaking, and then introduce it asynchronously on this basis.
Below is the comparison result of our optimized process:
In this step, we save resource download time and part of execution time by unpacking on demand and loading asynchronously
Resource preloading
In fact, we have found a problem of “main process serialization” in the analysis stage. The execution of JS is single thread, but the browser is actually multi-threaded, which includes asynchronous requests (fetch, etc.), so our further idea is to fetch Data and resource download in parallel through multi-threads.
According to the current situation, the logic of interface fetching is generally coupled in the business logic or data processing logic, so the step of decoupling (decoupling from UI, business module, etc.) is essential. The pure FETCH request (and a small amount of processing logic) is stripped out and put to the stage of higher priority to initiate the request. So where do we put it? We know that the browser has priority for processing resources, normally in the following order:
- HTML/CSS/FONT
- Preload/SCRIPT/XHR
- Image/Audio/Video
- Prefetch
To do this in parallel, it is necessary to advance fetching to the first priority (to be executed as soon as the HTML is parsed, rather than waiting for a request to be made during the execution of the SCRIPT tag resource loading). Our process would look like this:
One important thing to note is that since JS execution is serial, the fetch logic must be executed before the main process logic and cannot be placed in nextTick (such as setTimeout(() => doFetch())), otherwise the main process will continue to use CPU time and the request will not be issued
Active task scheduling
Browser also has priority strategy for resources, but it does not know our business level, exactly want what resources first loading/execution, which resources after loading/execution, so we jump out, if the implementation of the whole business level resource loading + / access process into a a small task, these tasks solely by ourselves to control the: Does packaging granularity, loading timing, and execution timing mean that CPU time and network resources are being maximized?
The answer is yes, but generally for simple projects, the browser’s own scheduling priority policy is sufficient, but for large and complex projects that require relatively extreme optimization, it may be necessary to introduce a “custom task scheduling” scheme.
In the case of Quick BI, our initial goal was to make the main content of the first screen appear faster. Therefore, CPU/ network allocation should be made according to our business priority in terms of resource loading, code execution and number fetching, such as: I want the “card drop-down menu” to start loading only after the main content of the first screen is displayed or when the CPU is idle (that is, lower priority, or even higher priority to start loading and displaying as soon as the user mouses over the card). As follows:
Here we have wrapped a task scheduler whose purpose is to declare a piece of logic to start executing after one of its dependencies (Promises) completes. Our flowchart changes as follows:
The yellow blocks represent the parts of the module that have been prioritized and degraded, which helps reduce the overall first screen time
TreeShaking
All above methods are mostly based on priority, in fact, in the era of front-end engineering increasingly complex (large project has the hundreds of thousands of lines of code), the birth of a more intelligent optimization scheme is used to reduce packet size, the idea is simple: tools depend on analysis, will not be out in reference to the code from the final product.
Sounds cool, and it works really well, but here are a few things that TreeShaking doesn’t even mention on its website — how often it fails:
Side effects
Side Effects usually express code that has an impact on a global (such as a window object) or the environment.
As shown in the example, the B code appears to be unused, but there is such code as console.log(B (1)) in its file that Webpack and other packaging tools are afraid to remove, so it will be driven in as usual.
The solution
Specify explicitly in the package.json or webpack configuration which code has sideEffects (for example sideEffects: [” **/*.css “]) and code without sideEffects will be removed
IIFE class code
IIFE the Immediately Invoked Function Expression
As shown in the figure, this type of code will cause TreeShaking to fail
The solution
Three principles:
- Avoid immediate function calls
- [Avoid] immediate new operations
- [Avoid] code that immediately affects the global
Lazy loading
We mentioned under “Load on Demand” that unpacking an asynchronous import would invalidate the TreeShaking, but here’s another case:
As shown in the figure above, because index.ts synchronizes the sharedStr import from bar.ts, and then asynchronously imports (‘./bar’) at the same time somewhere, this will cause two problems:
- Failure (TreeShaking
unusedStrWill be infiltrated) - Asynchronous lazy load failure (
bar.tsWill andindex.tsSmack them together)
When the amount of code reaches a certain order of magnitude, it is very easy for N people to co-develop this problem
The solution
- Avoid synchronous and asynchronous import of the same file
On-demand strategy (Lazy)
In fact, we have talked about some on-demand loading solutions, here we are appropriate to extend: since the load of the resource bundle can be on demand, whether a component rendering can be on demand? Can an object instance be used on demand? A data cache can also be generated on demand?
LazyComponent (LazyComponent)
As shown in the figure, Pierc. Private. Ts corresponds to a complex React component. Pierc encapsulates a lazy component by MakeRazyComponent as a default lazy component that will be loaded and executed only when the code is executed here. You can even declare a dependency with a second argument (deps) that is loaded and executed only when the dependency (promise) is completed.
LazyCache
Lazy caching is used in situations where the result of a transformation of some data in a data stream (or other subscriptable data) needs to be used anywhere and only at the moment of use
Lazy object (LazyObject)
A lazy object means that the object is instantiated only when it is used (properties/methods accessed, modified, deleted, etc.)
As shown in the figure, when GlobalRecorder was introduced, it was not instantiated and was only instantiated when GlobalRecorder.record() was called
Data streaming: Throttled rendering
For the convenience of state management in medium and large projects, the scheme of data flow is usually used, as follows:
The data stored in store is usually atomic with very small granularity. For example, in state, there are: A, B, C… If N=20, then there will be 20 View updates in 16ms (1 frame). If N=20, then there will be 20 View updates in 16ms (1 frame) :
This can cause significant performance problems. Therefore, we need to buffer the number of actions for a short period of time, and only update the View once after the 20 actions have changed state, as follows:
This solution works well in Quick BI in the form of Redux middleware in complex + frequent data update scenarios
thinking
“The gentleman preempts the problem by thinking about it”. When we look back, more than 80% of these performance problems can be avoided in the architectural design and coding stage, and 20% can be balanced by “space <=> time replacement strategy” and other ways. Therefore, the best performance optimization depends on our dedication to the quality of each piece of code: do we take into account the size of the build product caused by such module dependencies? Are you considering how often this logic is likely to be executed? Do you allow for the controllability of space or CPU usage as the data grows? And so on. There is no silver bullet for performance optimization. As a technical person, you need to internalize (know the underlying principles) the silver bullet of implanting performance into your instinctive thinking. You can click here to get a free trial of the product and experience Quick BI performance optimization.
Related products: Dataphin for intelligent data building and management
Data center platform is the only way for enterprises to become digital intelligent. Alibaba believes that data center platform is an intelligent big data system that integrates methodology, tools and organization, and is “fast”, “accurate”, “complete”, “unified” and “universal”.
At present, we are exporting a series of solutions through AliCloud, including general data mid-platform solution, retail data mid-platform solution, financial data mid-platform solution, Internet data mid-platform solution, government data mid-platform solution and other subdivided scenarios.
Among them, AliCloud data middle platform product matrix is based on Dataphin and started with Quick series as business scenarios, including:
- – DATAPHIN, one-stop and intelligent platform for data construction and management;
- – Quick BI, intelligent decision-making anytime and anywhere;
- – Quick Audience, all-round insight, global marketing, intelligent growth;
- – Quick A+, A one-stop data-based operation platform for multi-terminal and comprehensive application experience analysis and insight;
- – Quick Stock, intelligent goods operation platform;
- – Quick Decision, intelligent Decision platform;
Official site:
Data on the official website https://dp.alibaba.com
Nail communication group and WeChat public account
Copyright Notice:The content of this article is contributed by Aliyun real-name registered users, and the copyright belongs to the original author. Aliyun developer community does not own the copyright and does not bear the corresponding legal liability. For specific rules, please refer to User Service Agreement of Alibaba Cloud Developer Community and Guidance on Intellectual Property Protection of Alibaba Cloud Developer Community. If you find any suspected plagiarism in the community, fill in the infringement complaint form to report, once verified, the community will immediately delete the suspected infringing content.