Component reuse dilemma

In the process of moving bricks, when we get a design for a new requirement, we often look to see if there are components that have been written that can be reused in order to speed up development. If you can find a similar component, there’s a good chance you can tinker with it and use it directly.

For the projects I develop most often, when I look at the design draft, I can immediately remember where a reusable component is located in the project. However, as the project grew from 10+ route configurations to 100+ or even 200+ configurations, it was not enough.

Another problem is that other students on the team may have developed components that are very similar to the design draft, but because you are not familiar with other people’s projects, you cannot quickly identify which components are usable. Even if you know which component it is, the component may have a bunch of utility code dependencies, and to move the component into the target project in its entirety, you need to copy the component dependencies along with it. Your colleagues may be busy online and unable to respond to you in a timely manner, which can result in high communication costs.

Zhuan is a second-hand e-commerce platform, if you carefully observe the page UI of each line of business, you can find that most of them follow the same UI style. In fact, many e-commerce platforms have similar UI structures. Below is the front page of the three business lines. If you look closely, you will find that they have many similar places: the search box at the top, king Kong position, rotation chart, operation card, and commodity card. Although they are very similar, there are slight differences in some details, and these pages are managed by different front-end students, resulting in similar functions being implemented by different members many times.

So can we add some automated processes to this problem to improve the code reuse rate and enhance the happiness of brick moving?

The project design

First, clarify the purpose: after the emergence of new requirements, let front-end students know whether there are similar components, and quickly get the corresponding code files.

The problem then becomes how to compare the design draft to existing components and find those that are similar. If you want to compare, you have to have quantitative standards. Inside Sketch, designers primarily use sketch as a design tool, and inside Sketch, data is stored in JSON format. If you change the.Sketch file to.zip and unzip it using a compression tool, you’ll find that it actually stores a lot of JSON-formatted data that describes the nesting relationships between nodes and the styles of the nodes. Then an algorithm can be adopted to produce a quantitative standard that can be compared based on the data in the Sketch file, and the design draft can be processed.

So what do you do with components that already exist in each project? In fact, for these components, as long as you get the DOM structure they actually render, you can extract a feature data from the DOM structure. With the data of the design draft and the existing components, as long as the follow-up comparison of the two pieces of data, calculate the similarity, you can quickly find the component code needed for front-end development. For the above illustration, an analogy can be made to help understand the hashing algorithm. JS often uses objects to store data in the form of key-value. The hash algorithm produces a unique result based on a key value, and stores the data at the corresponding address based on that result. This process of generating results based on keys is similar to the node/DOM structure of the design draft above, producing feature data that can be compared.

Suppose that according to a design draft, five similar components have been found, how to let the front-end students know which of these components is the one he needs most? It would be too slow for him or her to look at the code for each component before deciding. So take a screenshot of each component, and when you see a screenshot of each of the five components, the one that looks the most like the design is basically the one he or she needs.

A summary of the scheme is: ** “According to the design draft and the DOM of the existing components, produce a data for comparison, initially screen out several similar components, and then let the business development according to the screenshots and codes, select the appropriate components in the initial screening of components, import into their own projects. **

Architecture design

The whole involves four parts:

  • The browser runtime analyzes the component’s DOM features, screenshots, and passes the results to the Vite plug-in.

  • Vite plug-in

    • Extract all dependencies of the component and publish them as NPM packages.
    • Integrate DOM features and screenshots and upload data to the server.

  • NPM server

  • Server side (Ma Liang background) : store data to MySQL. The detailed data flow is shown in the figure below. “Component hash” in the figure has the same meaning as “component characteristic data” above.

Here are a few questions this chart might raise:

Q: How do Vue components recurse in browsers?

A: Vue places the component instance on the dom’s __vue__ property during the mount process. Since the convention for Vue applications is to mount the root component to #app, we can get the root component of a Vue application by const root = document.getelementByid (‘app’).__vue__. All Vue components are then accessed recursively through root.$children.

Q: How do I take screenshots of components?

A: The screenshot solution considers Puppeteer and HTML2Canvas. In this scenario, HTML2Canvas is more suitable. Html2canvas may not be as powerful as Puppeteer, but it has the advantage of being able to generate screenshots of the specified DOM. Assuming that there is a full-screen popover in the page, covering all DOM elements, HTML2Canvas can still exclude the influence of popovers and generate screenshots of any DOM elements in HTML.

Q: Why is there a Vite plugin?

A: A component has many dependencies, and when collecting component code, it must be collected along with its dependencies, so you need to build A dependency tree. There are two ways to build a dependency tree, either by analyzing all the import statements through the AST or by using the existing dependency tree of the build tool. The implementation of AST analysis has many difficulties, such as aliases in import statements and numerous file types (less, SCSS, Vue, JS). If you want high accuracy, you need to parse the alias configuration of the build tool and use the AST transformation library for each file type. This is much more difficult to implement than with the help of build tool capabilities. What remains is a choice between the Vite plugin and the Webpack plugin. Since I used to access Vite for the main project of the team before, I am more familiar with it than Webpack, so I choose Vite plug-in as the means of relying on collection.

Q: How does the browser runtime code communicate with the Vite plug-in?

A: Vite2.0 uses the Connect framework as the HTTP Server, which is very similar in usage to Express and supports the same middleware syntax. Once the necessary middleware, such as Body-Parser, is introduced for Connect, you can add interfaces to it just as you would server-side code.

Traversal components

The purpose of this step is to get three data:

  • Component screenshots
  • Component characteristics
  • Component path in project screenshots and features are easy to understand, but why do you need component path in project? This requires attention to the environment in which the code is running. Because you need to access DOM data, this code must be run in a browser. But how does a component instance retrieved in a browser relate to a project instance.vueWhat do the files correspond to? The answer is in the development environment, on the Vue instancevm.$options.__fileProperty is the relative path of the Vue file in the project (figure below).But where does this property come from? Read the plugin and Vue source code to see why. (The last two paragraphs contain Vue source code analysis, which you may skip if you are not familiar with.) Support for Vue2 in Vite is provided byvite-plugin-vue2In its source code, there is a paragraph like this:
// Expose filename. This is used by the devtools and Vue runtime warnings.
if(! options.isProduction) {// Expose the file's full path in development, so that it can be opened
  // from the devtools.
  result += `\n__component__.options.__file = The ${JSON.stringify(
  path.relative(options.root, filePath).replace(/\/g, '/` '))}}Copy the code

As you can see, the plugin exposes the component’s path relative to the root directory in __component__.options.__file in non-production mode. Here __Component__ is the Vue component constructor. Those familiar with the Vue2 source code should know that Vue2 implements a similar function to the ES6 extends keyword via vue. extend. In the Vue subcomponent constructor there is this logic:

function initInternalComponent (vm: Component, options: InternalComponentOptions) {
  // vm.construct is the return value of vue. extend.
  The prototype for vm.$options is the options object of the constructor.
  const opts = vm.$options = Object.create(vm.constructor.options)
  // omit the following code
}
Copy the code

To sort out the process, it is:

  • vite-plugin-vue2To the constructoroptionsProperty injection file relative path in the project.
  • vm.$optionsThe prototype of an object is a constructoroptionsObject.
  • vm.$optionsItself has no__fileProperty, but is accessed by the stereotypeoptionsThe object of the__fileProperties. So in the browser environment, you can passvm.$options.__file“Asked the visitor.vite-plugin-vue2Relative path of the injected file.

Here is the pseudocode for traversing a Vue component instance:

// Vue root component instance
const appInstance = document.querySelector('#app').__vue__
/ * *@type {import('vue-router').default} * /
// Get the Vue Router instance
const router = appInstance.$router
// Vue component instance that the current route matches
const pageVM = router.currentRoute.matched[0]? .instances? .defaultif (pageVM) {
  traverseAppInstance(pageVM)
}

function traverseAppInstance(vm) {
  // Capture pseudo code
  const image = html2canvas(vm.$el)
  // Component feature analysis pseudocode, see the specific algorithm below.
  const characteristic = analyzeComponent(vm.$el)
  const filePath = vm.$otpions.__file

  // Pass the data to the Vite plug-in
  sendToVitePlugin({
    image,
    characteristic,
    filePath
  })
}
Copy the code

Feature extraction and comparison algorithm

❝

In the actual implementation process, d2C (Design to Code) platform (a platform that can transform Sketch files into well-organized codes) was used in order to achieve the effect quickly, but this did not decisively affect the overall implementation of the idea in this paper.

❞

To know which modules in the design draft are similar to those in the component library, we need a comparison algorithm. In fact, the simplest scheme is similar image comparison.

Scheme 1: Comparison of similar images

Although we can easily find similar components by using the algorithm of image similarity contrast correlation, this scheme will have obvious defects in actual scenes: We are extracting components from real pages, but the data in the components have already used real business data, which is very different from the content of the design draft. As a result, the scheme of similar image comparison can hardly play a role, so scheme 1 is not recommended.

Scheme 2: Component feature comparison

We can compare the structural style characteristics of the draft-generated code to components, as shown here in an example.

The left side of the figure above is the module in the design draft, and the right side is the real component in the project. Our human brain will identify these two modules as similar modules according to natural thinking. What is this thinking process like?Does that make them more certain that they are similar?

Based on this simple abstract process, we implement feature comparison algorithm.

Step 1: Feature extraction

The actual development of any one module, the engineer may have multiple nested hierarchy methods to implement, but different people may have different nested design, so we need to filter out the dimension hierarchy, first by traversal we get all the leaf node of a DOM structure, which is at the bottom of the DOM node, and we usually, Leaf nodes may be of the following types:

  1. The text
  2. The picture
  3. background
  4. Style nodes with visual space, such as buttons, graphics, forms, etc

Type 4 is a little complicated. We first take type 1, 2 and 3 as examples. We need to calculate and extract the following features:

  1. The node type can be Text, img, or BGIMG
  2. Node key style:
  • Font related styles
  • Image related styles
  • Background image related styles
  1. The coordinate of each leaf node ** “relative to the center point of the component” **. Third, why do we extract the coordinate of the node relative to the center point, which involves the comparison algorithm:

Step 2: Compare algorithms

The overall idea of feature comparison algorithm is as follows:

  1. Compare the proportion of leaf nodes of similar types in two components
  2. Compare the proportion of each leaf node that has the same type and position in the other component
  3. Compare the proportion of leaf nodes with similar key styles, all of the same type and location
  4. A scoring algorithm is used to calculate the similarity scores of the two components
  5. Finally, the components with more than a certain score are selected as similar components through a tradeoff algorithm

The key point in idea 2 is the same location. In actual comparison, we will find that even the same component may have different sizes and relative positions on different pages. Let’s first enlarge the right picture of the skeleton screen above:

It can be clearly seen that although they are generally similar, their positions are almost different. Therefore, instead of using absolute positions as a measurement standard, we can use coordinates relative to the center to measure:

image-20220309155514143

We calculate the coordinates of each leaf node relative to the center (offsetX, offsetY) and then scale the two components to the same width:

image-20220309155945056

Is it easier for us to compare relative positions? Of course the actual algorithm is much more complicated than that. If you are careful, you will find that the two components are not exactly the same. There is a HOT icon on the right side of the component:

To a certain extent will affect similar ratings, we all mentioned in the above algorithm idea, and we calculated in various conditions the same proportion, any condition, that is, we can know that each node is similar to another node and how much is not similar proportion respectively, then rely on the final score and comparison algorithm to determine the balance, In the above case, the actual score does not affect our judgment of similarity.

Vite plug-in

After the Vite plug-in gets the data sent by the browser, it uses filePath(the relative path of the file) to locate the specific.vue file and analyze its dependencies.

In Vite plug-in, you can obtain the Vite internal module dependency table, which is several maps, you can obtain the corresponding module through the file path.In an attempt tosrc/main.tsModules, for example,importedModulesismain.tsAll dependencies introduced by the file. Sent through the browserfilePathProperty to get the corresponding VUE file module. The Vue file introduces other files, which in turn introduce other files, so the module is really an N-fork tree. By traversing the n-fork tree, you can vue all the dependent files of the file. Once you have all the documents, passnpm publishCommand to be published as an NPM package. To use this component later, download the NPM package.

This process can be very detailed, such as a page with many components, if each component is traversed once, many components will be traversed repeatedly, resulting in unnecessary performance loss. By using the sequential traversal of binary tree, one traversal can be achieved and the respective dependencies of all VUE components can be collected. For example, the dependency files of each VUE component are different. NPM publish is usually used to publish a fixed project, and the scope of the publish file is constant. But the current scenario requires dynamic decisions about what to publish and many, many other details.

Nodejs server & MySQL

The server is where the data is ultimately stored, including all data collected during the process, including screenshot urls, component characteristics, NPM package names, and so on. Because it’s just a simple CRUD, I won’t repeat it here.

Results show

After all the above, the front end students get a new design draft, upload it to the Maliang system, and make a similarity match with all the extracted component features, and recommend it to the front end students to use. The final form on the Maliang system is:

conclusion

In the whole process, the features and screenshots of components are extracted through the run-time code, the Vue component code and all dependencies are obtained through the Vite plug-in, and the integrated data is uploaded to the Nodejs server and stored in the database. Finally, in Maliang system, users upload a sketch design draft, and recommend similar components to users by comparing the similarity between existing components and the design draft.

It’s much faster for users to make changes on a file that already has templates and CSS written than to start from scratch. And it breaks down the code sharing barrier between different projects, making the development of business pages smoother.

Thank you

The scheme mentioned in this paper is jointly completed by @Zhang Suoyong (component feature extraction and comparison) @Qiang Min (browser side code) @Chen Yitao (Vite plug-in).