Doupi fans, meet again. In today’s issue, Feng Gong from Bytedance data Platform will take you to explore the virtualization principle and scheme of Table component in depth.

Author: Feng Gong

preface

Table and table virtualization optimization is not a new topic. Recently, the team found that the industry does not have a relatively permanent solution to table virtualization. Why is that? And how to solve it? In this paper, we will introduce the different practical ideas of Table component virtualization in the team under React+AntDesign technology stack step by step, and analyze the possible difficulties.

Virtualization Issues

In list pages, waterfall flows, and Select components, we are likely to encounter scenarios that render large orders of magnitude lists. In the past, we’ve come up with an effective way to do this: for lists, we calculate to make sure that only part of the elements are rendered at a time as we scroll through the window. This not only reduces the pressure of the first screen, but also ensures that the long loading time does not have more performance burden, which can meet the performance optimization requirements of most of the above scenarios. Specifically, we use the known fixed row height and scrolling offset to calculate the table row index to scroll, only render the elements needed in the finite window, and set the list accordingly, the process is described as follows:

  • StartIndex to calculate the start data of the current visible region
  • Computes the endIndex of the current visible region end data
  • Calculates the data for the current visible area and renders it to the page
  • Calculate the startIndex data offset startOffset in the entire list and set it to the list
  • Calculate endOffset, the offset position of endIndex data relative to the bottom of scrollable area, and set it to the list

This scenario is widely discussed by most developers. I recommend to refer to the implementation principle of shallow virtual list, he thought of the problem, implementation, have a more detailed description. Now, our scenario comes to the Table component of AntDesign. We face a very similar problem: in business, when Table involves 1000+ rows &100+ columns rendering, the page rendering time often needs around 5000ms due to the complex logic in cells. This is clearly unacceptable. In fact, from the long list to the Table component, our list simply went from a one-dimensional axis to a two-dimensional plane. Therefore, the so-called Table virtualization, is nothing more than the hope that the Table can achieve: in the compatible with the existing functions of Table, the realization of the Table only renders the window flat content, to hide the row and column outside the window.

Present situation

AntDesign

Our internal React + UMI + AntDesign stack is a common architectural foundation in the front-end world. AntD is a common react component library. It provides a Table component that can help us solve many common requirements, including but not limited to: row selection, row expansion, row filtering, data paging, column fixation… At present the AntDesign [email protected]@4.11Two major versions, the latter after some refactoring and optimization, are the latest official recommendations. The AntDesign3 documentation has removed the demo for virtualization support (it can be used in practice), and in the AntDesign4 documentation, you can see the official recommendation for users to access react-window to solve the problem of virtualization tables.From the official Demo, AntD provides a Components property that implements requirements by passing in an object to a virtualization component provided by ReactWindow in its body property

 ...
 // VariableSizeGrid is a component provided by react-window
 const renderVirtualList = (rawData, { scrollbarSize, ref, onScroll })=>
     <VariableSizeGrid 
       {...props}
     />
 ...
 <Table
    {...props}
    className="virtual-table"
    columns={mergedColumns}
    pagination={false}
    components={{
      // overwrite the body set by AntD
      body: renderVirtualList,
    }}
  />
  ...
Copy the code

ReactWindow

React-window is a popular open source library for solving table virtualization problems. It produces virtualization components with different features so that users can focus on solving problems in different virtual scenarios.

The principle of React-Window is not complicated. It is mainly the implementation of virtualization requirements described in this paper. It mainly adjusts the offset of the horizontal axis of the table dynamically by monitoring onScroll, extracts an appropriate amount of partial data, and renders the content of the visual area.

His predecessor was React – Virtualized. After refactoring and upgrading, the authors abstracted the table and list scenarios better, achieved better performance by reusing common logic, and reduced the code package size by 20%.

Problems encountered during access

It seems that AntD has come up with a virtualization solution. However, it is not hard to find that in Github’s Issue, users’ feedback on the official recommendation scheme is not satisfactory. Such as:

  • In problem #21022, function points such as expansion and multiple selection are missing after AntD3 is used.
  • In issue #20339, most of the table function points are missing after using AntD4.

For the above problems, the official reply is for users to solve by themselves, therefore:

According to AntD documents, virtualization is used for configuration. Although virtualization can be achieved to a certain extent, it will cause the loss of many table functions.

This is our main problem at the moment.

rc-table

Since no good virtualization scheme was provided by the authorities, we had to have a deep understanding of AntD’s internal architecture to find the breakthrough point of the problem. After reading the code, we can know that the AntD/Table code architecture in 4.11 is briefly described as follows:

  • In AntDesign/Table:

    • Initialize table size and column contents
    • Preliminary collation of events and data
    • Sorting, paging, filtering, and other functions to calculate the logic of data and column
    • Invoke the RC-table dependency

  • In the rc – table:

    • Registers all kinds of row and column cell events
    • Fulfill various style requirements such as column fixation and row expansion
    • Complete the rendering

So far, we can see that the AntD architecture itself has partitioned the table logic to a certain extent, and the logic unrelated to the order and content of data has been abstracted into the RC-Table library. Roughly speaking, we can understand it as the following figure:

We also have to think about:

  • These three layers of logic, do we preserve them, change them, or replace them?
  • If yes, how do I resolve the impact of virtualization logic on existing frameworks?
  • If not, how to choose a new framework?

Group plan display

There are many different ideas for table virtualization within the project group:

Solution 1: Implement virtualization based on RC-table, independent of React-Window and AntD

  • Implementation idea:

In different businesses, function points of table components we need are not consistent. In extreme scenarios, we may only need to use a small amount of AntD features with high customization. Therefore, we can consider giving up AntD, directly using RC-Table and making some modifications.

  • Implementation method:

    • Fork a stable version of rC-table into the code base
    • Modify the RC-Table based on virtualization principles
    • Self-implementation of sorting, selection, column fixation and other upper function points

  • Advantages and disadvantages of the scheme:

    • Advantages:

      • Instead of implementing the basic table rendering ourselves, which is done by RC-Table, we only need to make minor changes to the scrolling events for virtualization.
      • External features facilitate customization.

    • Disadvantages:

      • The functional basis of AntD is lost.

Scheme 2: intercept display data in AntD, manually destroy out-of-table DOM, and manually create placeholders.

  • Implementation idea:

When services rely heavily on AntD functions, AntD cannot be removed. We can only keep the whole framework and find a way to make some changes. So modify the table Scroll event in antD and use the new onScroll logic:

  • Determine the DOM row position and evaluate index
  • After the setState action is complete, manually destroy the DOM content outside the window, while creating an equal height blank area to maintain the scrollbar position.
  • Cut and update data content to ensure the correctness of data in Windows.

In this logic, only the data content passed in by AntD is modified. Other operations are completed manually based on JsDOM, which has a small impact and high completion degree. However, you need to carefully test the impact on individual table function points.

  • Advantages and disadvantages of the scheme:

    • Advantages:

      • Forward compatible Antd Table configuration parameters, only need to add a few props, very low transformation cost
      • Direct control of DOM, development work in addition to data interception, most of the rest do not rely on AntD/rcTable code, performance is guaranteed.

    • Disadvantages:

      • You need to be careful how the change affects features such as column fixation and row expansion.
      • Since the row height cannot be obtained in advance, direct positioning is not supported at present. The equivalent solution is to search the corresponding data first and then load the result into InfinityTable

Scenario 3: Re-implement the table

  • Implementation approach

When the project is highly customized, consider abandoning ANTD outright. Scheme 3 retained the definition of AntD props to ensure the convenience of migration from AntD. Then, react-Table was used to complete most functions, and react-Window was used for virtualization. A new basic table with virtualization functions was developed at the bottom.

  • Reasons for framework selection

React Hook framework react-Table, a popular open source framework on Github, is introduced in this solution. It is a data logic hook framework. Because the data logic such as sorting, selection and so on is similar, there is no need to rewrite it, which helps save the cost of data processing and does not interfere with UI and virtualization.

  • Advantages and disadvantages of the scheme:

    • Advantages:

      • Deep transformation, high degree of reconstruction, easy to do any subsequent extension, convenient performance optimization
      • Using open source products to realize the original AntD development content, saving some costs

    • Disadvantages:

      • The cost of basic table implementation is high. The basic table needs to be re-implemented with the API of React-Table

Reconstruction of key issues

Blank flashing

  • Problem statement:

You can see a description of this problem in the React-Window README. After an application is virtualized, the scroll action is too fast. As a result, the placeholder is not updated and only blank content is displayed, and the application is loaded after a short period of time. When scrolling through a table rapidly in succession, a state of constantly blinking blank content appears.

  • Problem solving:

    • Currently, there is no good solution. Increasing the size of the preload area and optimizing the cell rendering content can alleviate the severity of the problem.
    • Some students proposed to dynamically adjust the size of preload area by monitoring the speed of scroll, which is an unpractical idea.

The cell ADAPTS to line breaks

  • Problem statement:

When cell text content is long, you want the height to be adaptive. This requirement may at first appear to be addressed with the react-Window variable height component, but in reality, this component needs to provide a height function:

// Returns the size of a item in the direction being windowed. 
// For vertical lists, this is the row height. 
// For horizontal lists, this is the column width.
itemSize: (index: number) => number
Copy the code

When the text changes, we also need to render to get the height of each line, so the API isn’t as nice as it should be. This problem can only be solved by using the dom node ref after the second rendering, but this will slow down the energy, so after consideration, this solution is not supported

Column fixed

  • Problem description

In scenario 3, since the flex layout of div is used instead of the native table, in order to achieve column fixation, we need to use three tables to fix the effect, so when scrolling, we need to change the scrollTop of multiple tables and their data interception simultaneously.

  • Problem solving

    • You can use a single state to synchronize scrolltops between multiple tables, but this implementation can cause performance issues with table rendering sequencing, resulting in three table pairs not matching. (AntD’s Header and body alignment itself has this problem)
    • AntD does not distinguish between three tables, but uses the native TR/TD and the sticky feature of CSS in one table, which partially avoids this problem, but does not solve it well. In the case of fixedColumn, the synchronization of header and body still has similar problems.
    • You can change the three tables into one table, and then use the highest-performing 3Dtransform to solve this problem. The open source product rsuite-table recommended below is a better solution to the problem through this solution.

Column virtualization

  • Problem Description:

    • As the current scheme focuses on the scenario with large amount of row data, performance problems still exist in the scenario with large amount of table column data.
    • If we introduce the React-Window Grid component for column virtualization, it will need to be compatible with sub-columns, column fixation and other functions, which is actually more complicated than expected.

  • Problem solving:

    • Excessive columns are not reasonable in terms of product design. It is suggested to improve data presentation at the product level.
    • Currently, column virtualization is not supported.

Open source table components are recommended

Since internal resources cannot be shared externally, there is no way to release the products of the above programmes. If you want to use tables with virtualization capabilities, but don’t want to do any development work, here are a few good open source products

  • rsuite-table

    • Rsuite. Making. IO/rsuite – tabl…
    • Virtualization and table performance is excellent and most feature points are complete
    • Weak header, no column filtering, no drag to change column width

  • react-base-table

    • Github.com/Autodesk/re…
    • Built-in virtualization, complete basic functions, other functions compared with AntD has a certain gap

conclusion

Table component virtualization is relatively simple in principle. Even without the help of React-Window, we saw that each student could use his or her own ideas to achieve similar functional requirements.

The reason for the present situation of numerous and ununified schemes is mainly due to the different requirements of different projects on the table library: some projects need low cost, some projects need to be compatible with various features, and some projects have heavy historical burden.

A new, all-inclusive Table library will have replacement costs, stability risks and other issues. Therefore, according to local conditions of their own projects to transform, to achieve a set of suitable for their own project virtualization scheme is the fastest, the most accepted.

Therefore, starting from three different scenarios, this paper summarizes different ideas of how to solve this problem within the scope of vision. For those who want to use it out of the box, several open source products are recommended for horizontal comparison.

It is better to teach people how to fish than to teach people how to fish. I hope the content discussed in this paper can more or less improve the performance of tables and the direction of table virtualization, give readers some inspiration, and bring faster and better experience for users of byte system products.

Data platform front-end team, responsible for the research and development of big data-related products such as Fengshen, TEA, Libra and Dorado. We maintain a very strong passion in front-end technology, in addition to data product related research and development, in data visualization, mass data processing optimization, Web Excel, WebIDE, private deployment, engineering tools are a lot of exploration and accumulation, you can contact us