Usage scenarios of virtual lists
If I want to put a large number of list items in the web page, pure rendering will be a great challenge to the browser performance, will cause scrolling lag, the overall experience is very bad, mainly have the following problems:
- Page waiting time is extremely long and user experience is poor
- CPU computing power is not enough, sliding will stall
- The GPU rendering capability is insufficient, and the page will jump
- The browser crashes due to insufficient RAM memory
1. Traditional practices
For a long list of rendering, the traditional way is to use lazy loading way, down to the bottom for new load content come in, is actually quite so paging overlay function in vertical direction, but as the load data is becoming more and more browser re-flow and re-paint overhead will be more and more big, the whole slide can cause caton, At this point we can consider using virtual lists to solve the problem
2. Virtual list
The core idea is to change only the rendering part of the list in the visible area when dealing with user scrolling. Specific steps are as follows:
StartIndex and endIndex are calculated. If the height of the element is fixed, the startIndex algorithm is very simple. StartIndex = math. floor(scrollTop/itemHeight), endIndex = startIndex + (clientHeight/itemHeight) -1, Then according to startIndex and endIndex, take the corresponding range of data, render to the visible area, and then calculate startOffset (upper scroll blank area) and endOffset (lower scroll blank area), the function of these two offsets is to support the contents of container elements, so as to play a buffer role. Keep the scroll bar scrolling smoothly and in a correct position
The above operations can be summarized in five steps:
- Instead of rendering all the long list data directly onto the page at once
- A portion of the long list is truncated to fill the viewable area
- The invisible parts of the long list of data are filled with blank placeholders (startOffset and endOffset areas in the image below)
- Listen for scroll events to dynamically change the visual list based on scroll position
- Listen for the scroll event to dynamically change the whitespace padding based on the scroll position
Virtual list of fixed height implementation steps
Nuggets are loaded in the traditional lazy way, not with a virtual list, here just want to express what is a fixed height list!
No matter how we scroll, all we change is the height of the scroll bar and the content of the elements in the viewable area. We don’t add any extra elements.
// Virtual list DOM structure<div className='container'>// The height of the box that listens for the scroll event inherits the height of the parent element<div className='Scroll box or else' ref={containerRef} onScroll={boxScroll}>// The box must be taller than the parent element, cannot scroll, and must dynamically change its padding value to control the state of the scroll bar<div style={topBlankFill.current}>
{
showList.map(item => <div className='item' key={item.commentId| | -Math.random() + item.comments)} >{item.content}</div>)}</div>
</div>
</div>
Copy the codeCalculate the maximum volume of the container
Simply put, we need to know how many list items can fit in the viewable area, which is one of the key steps before we intercept the content data and render it on the page
// The function that executes when the height of the scroll container changes
const changeHeight = useCallback(throttle(() = > {
// The height of the container is obtained by manipulating the DOM element because it is not necessarily a fixed value
curContainerHeight.current = containerRef.current.offsetHeight
// The maximum number of items in a list, considering that items may appear at the top and bottom of the list
curViewNum.current = Math.ceil(curContainerHeight.current / itemHeight) + 1
}, 500), [])
useEffect(() = > {
// The first time the component is mounted, the height and maximum capacity of the container are initialized
changeHeight()
// Since our visual window is dependent on the size of the browser, we need to listen for changes in the size of the browser
// When the browser size changes, you need to re-execute the changeHeight function to calculate the maximum capacity of the current visual window
window.addEventListener('resize', changeHeight)
return () = > {
window.removeEventListener('resize', changeHeight)
}
}, [changeHeight])
Copy the codeListen to scroll events dynamically intercept data && set up and down scroll buffering to eliminate fast scroll white screen
This is the heart of the virtual list, rendering only the elements we can see instead of all the elements we request, greatly reducing the number of DOM nodes in the container.
But there is a hidden problem we need to consider, when user sliding fast, many users of the equipment performance is not very good, it is easy to appear the screen has been rolling in the past, but the list items have not loading up in time, this time the user will see a short white, very bad user experience. So we need to set up a buffer area, so that users can still see the pre-rendered data after scrolling too fast, and when the buffer data is rolled out, our new data is also rendered to the page!
const scrollHandle = () = > {
// Note that this corresponds to the index value of the first element in the viewable area, not the number of elements
let startIndex = Math.floor(containerRef.current.scrollTop / itemHeight) // itemHeight is the height of each item in the list
// optimization: If user scrolling is triggered and the startIndex value is the same both times, there is no need to perform the following logic
if(! isNeedLoad && lastStartIndex.current === startIndex)return
isNeedLoad.current = false
lastStartIndex.current = startIndex
const containerMaxSize = curViewNum.current
/** * Solve the problem of sliding too fast white screen: Note that endIndex needs to be computed before startIndex is artificially changed * because we actually need three boards of data for low-performance devices to be used as a buffer area for scrolling up and down to avoid a white screen when sliding * startIndex is now the first element index in the viewable area. Add 2 times the number of viewable elements and you have one more board just below to use as a buffer */
// endIndex is used to create an extra board of data below the viewable area
let endIndex = startIndex + 2 * containerMaxSize - 1
// Near the bottom of the screen, the request is sent, and the bottom is not the last element of the viewable area, but the last element of that pane
const currLen = dataListRef.current.length
if (endIndex > currLen - 1) {
// Update the request parameters, send the request to get new data (but make sure that you are not currently in the request process, otherwise the same data will be requested repeatedly)! isRequestRef.current && setOptions(state= > ({ offset: state.offset + 1 }))
// If you have reached the bottom, set endIndex to the last element index
endIndex = currLen - 1
}
// endIndex is used to create an extra board of data at the top of the viewable area
// The startIndex value is artificially adjusted so that there is an extra board above the viewable area as a buffer, so that it does not slide to the bottom and request too slowly to get stuck at the bottom
if (startIndex <= containerMaxSize) { // containerMaxSize is the container capacity we calculated earlier
startIndex = 0
} else {
startIndex = startIndex - containerMaxSize
}
// Use the slice method to intercept data, but remember that the index element corresponding to the second parameter is not deleted, but can only be deleted before it, so we need to increment the endIndex
setShowList(dataListRef.current.slice(startIndex, endIndex + 1))}Copy the codeDynamically set up and down blank placeholders
This is the soul of virtual lists. We have very little data in nature, usually only a few to a dozen pieces of data, and without doing something additional to the list, it is difficult to even generate a scroll bar, let alone allow the user to control the scroll bar.
We must somehow prop up the content area so that the scroll bar will appear properly. The approach I’ve taken here is to set the paddingTop and paddingBottom values to dynamically spread out the content area.
Why dynamic change? For example, as we slide down we’re constantly removing elements from the top of the list and adding elements to the bottom. Because the element we delete originally occupies a certain height, if we do not increase the paddingTop value, the current top element will make up for this space, and our original intention is that the top element is used for display, so the scroll bar will shake, and the list items in the visible area will also jump
// The following code will be placed before updating the list data, also in the boxScroll event
// Change the style of the blank fill area, otherwise the elements in the visible area will not match the scroll bar, which will not achieve smooth scrolling effect
topBlankFill.current = {
// The initial index is the index of the first element in the viewable area. The index number indicates the number of previous elements
paddingTop: `${startIndex * itemHeight}px`.// endIndex is the last element we render and may not be visible; Subtract the index of the last dataListRef element from the endIndex to get the number of elements that have not yet been rendered
paddingBottom: `${(dataListRef.current.length - 1 - endIndex) * itemHeight}px`
}
Copy the codeDrop – down sites automatically request and load data
In a real development scenario, we would not request 1W or 10W pieces of data at one time, such a long request time, users would have closed the page, but also optimize fart haha!
So in real development, we still need to combine the original lazy loading mode, wait until the bottom of the drop-down to load new data in the cache data, and then according to the scroll events we decide which part of the data to render on the page.
// Request more data when the component is just mounted and when the pull-down hits bottom
useEffect(() = >{(async() = > {try {
// Indicates that the request is currently in progress
isRequestRef.current = true
const { offset } = options
let limit = 20
if (offset === 1) limit = 40
const { data: { comments, more } } = await axios({
url: `http://localhost:3000/comment/music? id=The ${186015 - offset}&limit=${limit}&offset=1`
})
isNeedLoad.current = more
// Add the newly requested data to the variable that stores the list data
dataListRef.current = [...dataListRef.current, ...comments]
// isRequestRef must be set to false before boxScroll because this variable will be used inside boxScroll
isRequestRef.current = false
// The boxScroll function needs to be triggered again when the latest data is requested, because the data inside the container and the blank fill area may need to change
boxScroll()
} catch (err) {
isRequestRef.current = false
console.log(err);
}
})()
// In boxScroll, optiOSN will be changed as soon as it hits the bottom
}, [options])
Copy the codeThe scroll event requests the animation frame to be throttled
Virtual lists rely heavily on scrolling events, and given that the user may swipe quickly, we have to keep the events short when throttling optimizations, otherwise the screen will still be blank.
Here I did not use the traditional throttling function, but used the request animation frame to help us throttling, here I will not do specific introduction, want to know can see my other article juejin.cn/post/708236… Juejin. Cn/post / 684490…
// A throttling optimization was made with request animation frames
let then = useRef(0)
const boxScroll = () = > {
const now = Date.now()
/** * The wait time should not be set too long, otherwise it will slide into the blank space. * Because the interval is too long, the scrolling update event is not triggered, and the slide will slide into the padding-bottom blank area. The interval of rendering is 16.6ms, our interval had better be less than the interval of two rendering 16.6*2=33.2ms, generally around 30ms, */
if (now - then.current > 30) {
then.current = now
// Calling scrollHandle repeatedly to let the browser execute the function before the next redraw ensures that no frames are lost
window.requestAnimationFrame(scrollHandle)
}
}
Copy the codeOf course, there are other ways to fill a blank area and simulate a scrollbar, such as having a box spread out the parent box to generate the scrollbar based on the total amount of data, calculating the distance from the viewable area to the top based on startIndex and adjusting the transform property of the content area element. StartOffset = scrollTop – (scrollTop % this.itemSize) so that the content area is always exposed in the visible area
So far, we have implemented the ability to display fixed height list items in virtual lists! Next we’ll look at how list items with variable heights (whose heights are stretched by content) can be optimized with virtual lists
Variable virtual list implementation steps
Micro-blog is a very typical not high virtual list, you can go to see if you are interested in oh!
In the previous implementation, the height of the list items was fixed, and because the height was fixed, it was easy to get the overall height of the list items as well as the scrolling display data and the corresponding offset. In practice, when the list contains variable content such as text and images, the height of the list items will be different.
We really have no way to know the height of each item before rendering the list, but we have to render it, so what do we do?
There is a solution, is to give no renders the list item set an estimated height, wait for the data into a true dom elements, and then get their true height to update the original set of estimates height, let’s look at what’s the difference between a list with fixed high, specific how to achieve it!
Request to new data to initialize the data (set the estimated height)
The setting of the estimated height is actually a skill. The larger the estimated height of the list item is, the less data will be displayed. Therefore, when the estimated height is much larger than the actual height, it is easy to appear that the amount of data in the visible area is too small and some blank space will appear in the visible area. To avoid this, our estimated height should be set to the minimum generated by the list items, so that there is no white space in the first screen shown to the user, even though it may render a few more pieces of data
// Request more data
useEffect(() = >{(async() = > {// New requests can be sent only when the request is not currently in the request state
if(! isRequestRef.current) {console.log('Request sent');
try {
isRequestRef.current = true
const { offset } = options
let limit = 20
if (offset === 1) limit = 40
const { data: { comments, more } } = await axios({
url: `http://localhost:3000/comment/music? id=The ${186015 - offset}&limit=${limit}&offset=1`
})
isNeedLoad.current = more
// Retrieves the index of the last data in the cache, or -1 if not
const lastIndex = dataListRef.current.length ? dataListRef.current[dataListRef.current.length - 1].index : -1
// Add the requested data to the cache array
dataListRef.current = [...dataListRef.current, ...comments]
const dataList = dataListRef.current
// Process the new data just requested by adding the corresponding index value, estimated height, and distance from the top to the end of the element
for (let i = lastIndex + 1, len = dataListRef.current.length; i < len; i++) {
dataList[i].index = i
// The estimated height is the minimum height corresponding to the list item
dataList[i].height = 63
// The distance between the head of each list item and the top of the container is equal to the distance between the tail of the last element and the top of the container
dataList[i].top = dataList[i - 1]? .bottom ||0
// The distance between the tail of each list item and the top of the container is equal to the distance between the head of the previous element and the top of the container plus the height of its own list item
dataList[i].bottom = dataList[i].top + dataList[i].height
}
isRequestRef.current = false
boxScroll()
} catch (err) {
console.log(err);
} finally {
isRequestRef.current = false}}}) ()// eslint-disable-next-line
}, [options])
Copy the codeThe estimated height in the cache is updated with the actual height of the list item after each list update
The React function uses useEffect to perform the function componentDidUpdate life cycle function, as long as the second parameter is not passed. As long as we re-render the list component, the real height of each item in the list is recalculated and updated to the cache. The next time we use the data in the cache, we will use the real height
// Every time the component rerenders, i.e. the user scrolls the data, we need to update the list item heights that we do not know yet into our cache data so that the next update can render normally
useEffect(() = > {
const doms = containerRef.current.children[0].children
const len = doms.length
// Since we did not request data at the beginning, there is no need to perform further operations even if the component is rendered but there are no list items
if (len) {
// Iterate over all the nodes in the list, changing the height in the cache according to the actual height of the node
for (let i = 0; i < len; i++) {
const realHeight = doms[i].offsetHeight
const originHeight = showList[i].height
const dValue = realHeight - originHeight
// If the actual height of the list item is the height in the cache, no update is required
if (dValue) {
const index = showList[i].index
const allData = dataListRef.current
/** * If the actual height of the list item is not the height in the cache, then not only the bottom and height attributes of this item in the cache will be updated * and all subsequent items in the cache will be affected by it, so we need another layer of for loop to change the subsequent values in the cache */
allData[index].bottom += dValue
allData[index].height = realHeight
/** * Note: Select * from 'showList' where 'bottom' and 'next top' are not contiguous. Because startIndex, endIndex and blank fill area are calculated according to the values of top and bottom *, the final calculation result will be wrong. The startIndex obtained by sliding varies greatly and the scrollbar is unstable, causing obvious jitter problem */
for (let j = index + 1, len = allData.length; j < len; j++) {
allData[j].top = allData[j - 1].bottom
allData[j].bottom += dValue
}
}
}
}
// eslint-disable-next-line
})
Copy the codeGets the start and end element index of the viewable area && sets the scroll up and down buffer area to eliminate the white screen of fast scrolling
The bottom attribute of the list item represents the distance from the end of the element to the top of the container. The bottom of the last element in the viewable area is the first one greater than (scroll height + viewable height). We can use this rule to traverse the cache array to find startIndex and endIndex
Since our cache data itself is sequential, the method of obtaining the initial index can be considered to reduce the number of retrieval times and time complexity through binary search
// Get the starting and ending indexes of the data to render
const getIndex = () = > {
// Set the amount of data in the buffer area
const aboveCount = 5
const belowCount = 5
// Result array, which contains the start index and end index
const resObj = {
startIndex: 0.endIndex: 0,}const scrollTop = containerRef.current.scrollTop
const dataList = dataListRef.current
const len = dataList.length
// Set the upper buffer. If the index value is larger than the buffer area, reduce the startIndex value to set the top buffer
const startIndex = binarySearch(scrollTop)
if (startIndex <= aboveCount) {
resObj.startIndex = 0
} else {
resObj.startIndex = startIndex - aboveCount
}
/** * The element in the buffer whose bottom is greater than the scroll height plus the viewable area height is the last element in the viewable area. We can only add */ to the last element in the cached data
const endIndex = binarySearch(scrollTop + curContainerHeight.current) || len - 1
// Increase endIndex to set a buffer below the scroll area to avoid the white screen caused by fast scrolling
resObj.endIndex = endIndex + belowCount
return resObj
}
// Since our cache data itself is sequential, so the method to obtain the start index can be considered through binary search to reduce the number of retrieval:
const binarySearch = (value) = > {
const list = dataListRef.current
let start = 0;
let end = list.length - 1;
let tempIndex = null;
while (start <= end) {
let midIndex = parseInt((start + end) / 2);
let midValue = list[midIndex].bottom;
if (midValue === value) {
// indicate that the current scroll area plus the visual area is exactly the boundary of a node, so we can use the next node as the end element
return midIndex + 1;
} else if (midValue < value) {
// Since there is a gap between the current value and the target value, we need to increase the start value to make the midpoint fall further behind next time
start = midIndex + 1;
} else if (midValue > value) {
// Because our goal is not to find the first value that satisfies the condition, but to find the smallest index value that satisfies the condition
if (tempIndex === null || tempIndex > midIndex) {
tempIndex = midIndex;
}
// Since we are going to continue to find smaller indexes, we need to make end-1 to narrow the range so that the midpoint falls further forward next time
end--
}
}
return tempIndex;
}
Copy the codeListen to scroll events dynamically intercept data && dynamically set up and down blank placeholders
The operation of dynamic data interception is almost the same as the virtual list of fixed height, but the big difference is in the calculation of the padding value. In the fixed height list, we can directly calculate the height of the blank fill area based on the starting and ending index values.
In fact, in the indeterminate list, the calculation method is easier, because the top value of the element corresponding to startIndex is the upper blank area that we need to fill, and the lower blank area can also be calculated by the difference between the height of the whole list (the bottom value of the last element) and the bottom value of the element corresponding to endIndex
const scrollHandle = () = > {
// Get the starting and ending indexes of the current element to render
let { startIndex, endIndex } = getIndex()
/** * If the user scrolls, and the startIndex value is the same both times, then there is no need to perform the following logic, * unless the user requests the function again by default, which is a special case where the startIndex has not changed, but needs to perform subsequent operations */
if(! isNeedLoad && lastStartIndex.current === startIndex)return
// After rendering once, isNeedLoad needs to be initialized
isNeedLoad.current = false
// Monitor the value of lastStartIndex in real time
lastStartIndex.current = startIndex
// When the last element of the lower buffer area touches the bottom of the screen, the request can be sent
const currLen = dataListRef.current.length
if (endIndex >= currLen - 1) {
// Request parameters can be changed to send a request for more data only if the request is not currently in the request state! isRequestRef.current && setOptions(state= > ({ offset: state.offset + 1 }))
EndIndex cannot be greater than the index of the last element in the cache
endIndex = currLen - 1
}
// Blank fill area style
topBlankFill.current = {
// Change the style of the blank fill area. The top value of the starting element represents the distance from the top, which can be used as the paddingTop value
paddingTop: `${dataListRef.current[startIndex].top}px`.// The difference between the bottom value of the last element in the cache and that of the corresponding element in endIndex can be used as the paddingBottom value
paddingBottom: `${dataListRef.current[dataListRef.current.length - 1].bottom - dataListRef.current[endIndex].bottom}px`
}
setShowList(dataListRef.current.slice(startIndex, endIndex + 1))}Copy the codethinking
Although we have implemented a virtual list based on the dynamic height of the list item, but if the list item contains a picture, and the list height is supported by the picture. Network under this scenario, because the picture will send the request, the list items may have been rendered to the page, but the picture is not loaded, at this time there is no guarantee that we get the list items in real height have pictures when loading is completed, access to the height of the presence of contains picture height, leading to inaccurate calculation.
However, it is rare to see images arbitrarily spread the size of a box, as this would make the list very irregular. For example, the size of a picture, the size of 2×2 and 3×3 are all designed in advance. As long as we add a fixed height to the IMG tag, even if the picture is not loaded, But we also know exactly what the height of the list item is.
If you do have a scenario where a list item is stretched arbitrarily by the image, one way to do this is to bind the image to an onLoad event, wait until it is loaded, recalculate the height of the list, and then update it to the cached data. Secondly, you can also use ResizeObserver to monitor changes in the height of the content area of list items and obtain the height of each list item in real time. However, MDN says that this is only an experimental feature, so it may not be compatible with all browsers for the time being!
If you have a higher way, you can communicate in the comments section!