Problem analysis

Have you ever had a problem with girls as a programmer? Traditionally, the web is full of this kind of method. The client constantly updates the pagesize and PageIndex parameters according to its own scrolling, and then uploads the data to the server interface, and there is little on the web to say whether there is a problem with this method. Is there a problem?

When it comes to paging, no matter how your program is written, the core action of paging is to get a piece of data based on where it starts and where it ends. No matter how complex your sorting rules are, the goal is always to get a contiguous piece of data in the total list. Whether you use the SQL statement page directly or use a search engine (such as ES), the result will be the next page of data displayed on the client side.

Of course, the interaction represented on the client UI can have many styles

In the case of waterfall flow or app section rolling display, or other cases where the total number of data is not needed, the server should not query the total number of data, and the presenter can take the data on the next page as the basis of whether to continue to pull the data on the next page.

Back to the topic, is there a problem if the client uses pagesize and PageIndex parameters for paging requirements? Of course there is, otherwise what’s the point of writing this article? I’m not a bullshit programmer

The problem

Here is the simplest and most basic SQL statement paging as an example, if the existing data in the database is

1,2,3,4,5,6,7
Copy the code

The sorting rule is in reverse order of size, i.e., the full list of data is:

7,6,5,4,3,2,1
Copy the code

Pagesize = 2, pageindex = 2, the correct result is 5,4. Well, that’s true, if the data hasn’t changed, but what if the data has changed, if I add a new data 8 now, the list data will change to

8,7,6,5,4,3,2,1
Copy the code

According to the above pagination principle, the data obtained on the second page is changed to “6,5”. You are smart enough to find the problem, which may also be the reason for D sister to cause overtime.

Paging is an operation based on dynamic data

To solve the problem

The data sources for paging operations are dynamic, and sometimes the part of the data that changes happens to be in the range of the data that you fetch can lead to data duplication or errors. So what’s the solution?

The client

As the demander and exhibitor of data, the client needs to remember the primary key list of loaded data. If a piece of data has been displayed, the client determines whether to display it again based on service requirements. In general, deduplication is required.

If the volume of data is very large, the client’s solution of maintaining a data pool is not ideal

The service side
  1. Add the last data ID parameter lastId to the server page and remove the pageIndex parameter, because in most cases, the pageIndex parameter is used to determine the starting point of the data on the server side. With lastId, pageinde is not needed in many cases.
  2. The server caches all the data, so that dynamic data is static for a certain period of time, but this is also a palliative.
  3. If there is no business requirement for sorting, the server can use sequential paging to place the retrieved data in unchanging data segments

It is difficult for a server to make dynamic data static

The business side

No amount of program optimization can change the ever-changing nature of data, and if the business (product, operation) can accept that data can occasionally be repeated, it can greatly reduce the work of programmers.

Sometimes what you consider data bugs in other business departments aren’t necessarily major issues

More interesting articles

  • Distributed large concurrent series
  • Architectural Design Series
  • Series of interesting algorithms and data structures
  • Design Pattern series