Redis paging cache method implementation

1. Set the cache directly. If there is a large amount of data, add, delete, or modify operations, the cache update frequency is high and the efficiency is low.

2. Set the cache by page number.

1. New - Delete all caches (sort backwards, insert data on the first page, change the list of subsequent pages), 2. Modify - Update the current page cache, 3. Delete - update the current page and subsequent pages cache.Copy the code

<? php class ArticleClass { private$pageCount= 10; // display each page /** * get the list * @param$page_no
     * @return array
     */
    public function getList($page_no) {$cache = getRedis();
        $cache_key = ArticleList;
        $list = $cache->get($cache_key);
        $count = $this->getCount(); // Count the total$page = new Page($count.$this->pageCount);
        $show = $page->show();
        if(!$list) {$list = M('Articles') - >limit($page->firstRow . ', ' . $page->listRows)->order('id DESC')->select();
            $ids = [];
            foreach($list as $value) {$ids[] = $value['id'];
                unset($value);
            }
            $oldids = $cache->get(ArticleId);
            $cache->set($cache_key.$list, 3600); // Cache for 1 hour$cache->set(ArticleId,$ids);
        }
        return ['page'= >$show.'list'= >$list]; } /** * get the total number of pages * @return mixed
     */
    public function getCount() {return M('Articles')->count(); } /** * get the last page number * @return float
     */
    public function getLastPage() {$count = $this->getCount();
        return ceil($count/$this->pageCount); } /** * get a record * @param$id
     * @return mixed
     */
    public function getInfo($id) {$info = M('Articles') - >where('id = '.$id)->find();
        return $info; } /** * Save information * @param$data
     * @return mixed
     */
    public function create($data) {return M('Articles')->add($data); } /** * Update information * @param$data
     * @param $id
     * @return mixed
     */
    public function updateById($data.$id) {return M('Articles') - >where('id = '.$id)->save($data); } /** * Delete information * @param$id
     * @return mixed
     */
    public function deleteById($id) {return M('Articles') - >where('id = '.$id)->delete(); } /** * delete/modify the cache * @param$type
     * @param null $page
     */
    public function delCache($type.$page=null){
        $cache = getRedis();
        $cache_key = ArticleList;
        if($type= ='add') {for($i= 1;$i< =$this->getLastPage();$i{+ +)$cache->rm($cache_key.$i);
            }
        }elseif($type= ='update') {$cache->rm($cache_key.$page);
        }elseif($type= ='delete') {for($i=$page;$i< =$this->getLastPage();$i{+ +)$cache->rm($cache_key.$i); }}}}Copy the code

Problem: ① Page number, may be artificially modified. ② Modify a data, change a batch of data.

3. One cache for one data

First mistake: Throw all data in cache first? No, the ids of all the data are stored in an IDS cache first. When the data is obtained, the ids are searched according to the IDS. If the data cannot be found in the cache, the ids are searched in the database through a certain ID and saved in the cache.

1. Use set to store content information by key+ ID, and store all data ids to another cache IDS, get IDS by interval, and get data by get key+ ID through for.

Questions raised:

The first person to access, when the cache is lost, to find all the records in the cache process, must be very slow!!Copy the code

Expire time will be added to the ids cache. Each record cache will lose data written to the cache when searching for the specified data. Permanently cache x, or set the valid duration. [The first person access slow problem, the subsequent implementation through the list, Redis list generally does not set the expiration time]Copy the code

public function getList() {$cache_ids = $this->Cache->get(ArticleId);
    $count = $this->getCount(); // Count the total$page = new Page($count.$this->pageCount);
    $show = $page->show();
    if(!$cache_ids){// This runs once in the background, permanently recording the cache$lists = M('Articles')->order('id DESC')->select(); //tp3.2 cannot directly find all data in a field$ids = [];
        foreach($lists as $value) {$ids[] = $value['id'];
            unset($value);
        };
        unset($lists);
        $this->Cache->set(ArticleId,$ids); // Record all ids to a cache // Currently need to fetch data$list = M('Articles') - >limit($page->firstRow.', '.$page->listRows)->order('id DESC')->select();
        foreach($list as $value) {$this->Cache->set($this->code_key.$value['id'].$value, 3600); // One cache per recordunset($value); }}else{
        $start = $page->firstRow;
        $end = $start+$page->listRows;
        for($i=$start;$i<$end;$i{+ +)if(empty($cache_ids[$i])){
                break;
            }
            $temp = $this->Cache->get(ArticleList.$cache_ids[$i]);
            if(!$temp){// a cache does not exist$temp = $this->getInfo($cache_ids[$i]);
                $this->Cache->set($this->code_key.$cache_ids[$i].$temp, 3600); // Record missing cache}$list[] = $temp;
            unset($temp); }}return ['page'= >$show.'list'= >$list];
}
Copy the code

Question 2:

Get all ids from foreach??Copy the code

Solution: 1. Use the array_column function to obtain the value of a column of the array and return a one-dimensional array. Filed (' ID ') can be added, and then a 2d array is stored. The sub-array has only one field ID. When it is filed, a segment of data is obtained through array_slice. Tips: Don't get caches through foreach!! Check each cache, not present, check database by ID, save cache again.Copy the code

public function getList() {$cache_ids = $this->Cache->get(ArticleId);
    $count = $this->getCount(); // Count the total$page = new Page($count.$this->pageCount);
    $show = $page->show();
    if(!$cache_ids){// This runs once in the background, permanently recording the cache$lists = M('Articles')->order('id DESC')->select(); //tp3.2 cannot directly find all data that returns a one-dimensional array as a field$ids = [];
        $ids = array_column($lists.'id');
        unset($lists);
        $this->Cache->set(ArticleId,$ids); // Record all ids to a cache$cache_ids = $this->Cache->get(ArticleId);
    }
    $start = $page->firstRow;
    $end = $start+$page->listRows;
    for($i=$start;$i<$end;$i{+ +)if(empty($cache_ids[$i])){
            break;
        }
        $temp = $this->Cache->get(ArticleList.$cache_ids[$i]);
        if(!$temp){// a cache does not exist$temp = $this->getInfo($cache_ids[$i]);
            $this->Cache->set($this->code_key.$cache_ids[$i].$temp, 3600); // Record missing cache}$list[] = $temp;
        unset($temp);
    }
    
    return ['page'= >$show.'list'= >$list];
}
Copy the code

Update the cache when adding, deleting, or changing

The following code has problems: 1. Add and modify does not consider the ids cache does not exist!! 2. And there are hidden dangers: when the user adds a large amount of concurrent data, the person inserted into the database first saves it in the cache slowly!! Cause list sort problem!! For example: current data 1,2,3; A inserts 4 first, but does not have time to save to the cached node! B inserts 5 and cache, causing the cache list that should be 1,2,3,4,5 to become 1,2,3,5,4 once and for all solution: add, delete, change the cache involved directly delete!! Because the cache can not be found when the access to the database will be checked and stored in the cache!!Copy the code

/** * Add a new cache * @param$id
 */
public function addCache($id) {$info = $this->getInfo($id);
    $this->Cache->set($this->code_key.$id.$info, 3600); // Record a cache$ids = $this->Cache->get($this->code_ids);
    array_unshift($ids.$id);
    $this->Cache->set($this->code_ids,$ids); } /** * Modify a cache * @param$id
 */
public function updateCache($id) {$info = $this->getInfo($id);
    $this->Cache->set($this->code_key.$id.$info,3600);//记录一条的缓存
}

/**
 * 删除一个缓存
 * @param $id
 */
public function delCache($id) {$this->Cache->rm($this->code_key.$id); // Record a cache$ids = $this->Cache->get($this->code_ids);
    array_splice($ids,array_search($id.$ids), 1);$this->Cache->set($this->code_ids,$ids);
}
Copy the code

The final code

<? php /** * Created by PhpStorm. * User: Administrator * Date: 2018/4/10 * Time: 14:41 */ namespace Api\Controller; use Think\Controller; use Think\Page; class ArticleClass { private$pageCount= 10; // Display private per page$Cache; // Store the cache object publicfunction __construct()
    {
        $this->Cache = getRedis(); } /** * get the list * @return array
     */
    public function getList() {$cache_ids = $this->getIdList(); // Get data // paging$count = count($cache_ids); // Count the total$page = new Page($count.$this->pageCount);
        $show = $page->show();
        $start = $page->firstRow;
        $end = $start+$page->listRows;
        $list = [];
        for($i=$start;$i<$end;$i{+ +)if(empty($cache_ids[$i])){
                break;
            }
            $temp = $this->getInfo($cache_ids[$i]);
            $list[] = $temp;
            unset($temp);
        }
        return ['page'= >$show.'list'= >$list]; } /** * get the list of ids * @return mixed
     */
    public function getIdList() {$cache_ids = $this->Cache->get(ArticleId);
        if(!$cache_ids{//id cache does not exist, come in$lists = M('Articles')->order('id DESC')->select(); //tp3.2 cannot directly return a one-dimensional array of all data for a field$ids = [];
            $ids = array_column($lists.'id');
            $this->Cache->set(ArticleId,$ids24 * 3600); // Record all ids to a cache$cache_ids = $this->Cache->get(ArticleId);
        }
        return $cache_ids; } /** * get a record * @param$id
     * @return mixed
     */
    public function getInfo($id) {$info = $this->Cache->get(ArticleList.$id);
        if(!$info){// a cache does not exist$info = M('Articles') - >where('id = '.$id)->find();
            $this->Cache->set(ArticleList.$id.$info, 3600); // Record missing cache}return $info; } /** * Add by id * @param$data
     * @return mixed
     */
    public function create($data) {$id = M('Articles')->add($data);
        $this->killActicleIdsCache();
        return $id; } /** * Update by id * @param$data
     * @param $id
     * @return mixed
     */
    public function updateById($data.$id) {$res = M('Articles') - >where('id = '.$id)->save($data);
        $this->kellActicleInfoCache($id);
        return $res; } /** * delete by id * @param$id
     * @return mixed
     */
    public function deleteById($id) {$res = M('Articles') - >where('id = '.$id)->delete();
        $this->kellActicleInfoCache($id);
        $this->killActicleIdsCache();
        return $res; } /** * remove the content cache * @param$id
     */
    public function kellActicleInfoCache($id) {$this->Cache->rm(ArticleList.$id); } /** * clear ids cache */ publicfunction killActicleIdsCache() {$ids = $this->Cache->get(ArticleId);
        if($ids) {$this->Cache->rm(ArticleId); }}}Copy the code

Mysql > select * from ‘list’; Get the data from the interval.

Disadvantages:

A. You can’t specify a key to change or delete. You need to rewrite the entire list (delete first, insert later).

B. Concurrent write cache, which may lead to data duplication (data is not unique, set)

List can be specified by lRE to delete each value, and list is also the storage id, information data is cached one by one. ②. Data duplication is mainly in the page turning time just someone write data (through the id reverse order), if the page current page number and pageSize each page display number of pages, will appear originally in the first page of the data, drop-down load second page when found also!! Because of the insertion of data in the header of the first page, the data at the end of the first page is pushed up to the second page or beyond…

Solution:

First, the data is looked up from tail to head in id order Rpush, and is fetched from tail to head. Paging through the index of the last record is worth going to the next pageStart value, and finally through lrange(ids,End) gets the paging interval. The index key is passed to the front end, which then returns it; If not, get the total llen(), get $end, and process. New insert tail via rpush() and delete specified value via lrem(). Update and modify Delete the corresponding INFO cache.

For example, 1-----5-----10-----15 There are 15 data pages on each page. Lrange (ids,14,14-5+1) is the data whose indexes are 10,11,12,13,14. (Index starts at 0)Copy the code

The complete code is as follows

class ArticleClass
{
    private $pageCount= 5; // Display private per page$Cache; // Store the cache object publicfunction __construct()
    {
        $this->Cache = getRedis(); } /** * get the list * @param null$keyList index * @return array
     */
    public function getList($key= null) {/ / pagingif($key == null){
            $end = $this->getArticleIdLen();
        }else{
            $end = $key;
        }
        $end = $end - 1;
        if($end< 0) {return ['list'= > [].'key'= > 0]; }$start = $end - $this->pageCount + 1;
        if($start< = 0) {$start = 0;
        }
        $cache_ids = $this->getRangeId($start.$end); // Get data$list = [];
        foreach($cache_ids as $id) {$info = $this->getInfo($id);
            $list[] = $info;
        }
        rsort($list);
        return ['list'= >$list.'key'= >$start]; } /** * get the ids cache length * @return mixed
     */
    public function getArticleIdLen() {if(!$this->Cache->exists(ArticleId)){// Cache does not exist$this->setIdList();
        }
        $count = $this->Cache->llen(ArticleId);
        return $count; } /** * get the ID interval cache * @param$start
     * @param $end
     * @return mixed
     */
    public function getRangeId($start.$end) {if(!$this->Cache->exists(ArticleId)){// Cache does not exist$this->setIdList();
        }
        $cache_ids = $this->Cache->lrange(ArticleId,$start.$end);
        return $cache_ids; } /** * Set the ID list cache * @return mixed
     */
    public function setIdList() {$lists = M('Articles')->order('id ASC')->select();
        $ids = [];
        $ids = array_column($lists.'id');
        foreach($ids as $value) {$this->Cache->rpush(ArticleId,$value); }} /** * Get a record * @param$id
     * @return mixed
     */
    public function getInfo($id) {$info = $this->Cache->get(ArticleList.$id);
        if(!$info){// a cache does not exist$info = M('Articles') - >where('id = '.$id)->find();
            $this->Cache->set(ArticleList.$id.$info, 3600); // Record missing cache}return $info; } /** * Add by id * @param$data
     * @return mixed
     */
    public function create($data) {$id = M('Articles')->add($data);
        $this->Cache->rpush(ArticleId,$id); // Insert the end of the listreturn $id; } /** * Update by id * @param$data
     * @param $id
     * @return mixed
     */
    public function updateById($data.$id) {$res = M('Articles') - >where('id = '.$id)->save($data);
        $this->kellActicleInfoCache($id);
        return $res; } /** * delete by id * @param$id
     * @return mixed
     */
    public function deleteById($id) {$res = M('Articles') - >where('id = '.$id)->delete();
        $this->kellActicleInfoCache($id);
        $this->Cache->lrem(ArticleId,$id); // Delete the ID list cachereturn $res; } /** * remove the content cache * @param$id
     */
    public function kellActicleInfoCache($id) {$this->Cache->rm(ArticleList.$id); // Delete a cache}}Copy the code

Extension problem: The amount of data is too large, through the time divided into multiple REDis storage, every month to save a list of REIDS, how to get the value of this month and then get the next month, and the data connection is good?

Solution: Obtain the data of the current month first when obtaining the value. If the data does not meet the number of pages to be displayed, obtain the next cache supplement. After obtaining a cache, obtain the next cache. The front end passes an extra cache block ID.

The code is as follows:

class ArticleClass
{
    private $pageCount= 5; // Display private per page$Cache; // Store the cache object private$cache_block = ['the 2018-04'.'the 2018-03'];
    public function __construct()
    {
        $this->Cache = getRedis(); } /** * get the list * @param null$keyList index * @return array
     */
    public function getList($key=null,$block= 0) {/ / pagingif($key == null){
            $end = $this->getArticleIdLen($this->cache_block[$block]);
        }else{
            $end = $key;
        }
        $end = $end - 1;
        if($end< 0){// If there is no value, take it to the next cache$block+ +;$end = $this->getArticleIdLen($this->cache_block[$block]);
            $end = $end - 1;
            if($end< 0) {$block-;return ['list'= > [].'key'= > 0,'block'= >$block]; }}$start = $end - $this->pageCount + 1;
        if($start< = 0) {$start = 0;
        }
        $cache_ids = $this->getRangeId($start.$end.$this->cache_block[$block]); // Get dataif(count($cache_ids) < $this->pageCount){// Finally return the number of insufficient, take down a supplement$block+ +;$end = $this->getArticleIdLen($this->cache_block[$block]);
            $end = $end - 1;
            if($end> 0) {$start = $end - $this->pageCount + 1 + count($cache_ids);
                $block_ids = $this->getRangeId($start.$end.$this->cache_block[$block]);
                $cache_ids = array_merge($cache_ids.$block_ids);
            }else{
                $block-; }}$list = [];
        foreach($cache_ids as $id) {$info = $this->getInfo($id);
            $list[] = $info;
            unset($id);
        }
        rsort($list);
        return ['list'= >$list.'key'= >$start.'block'= >$block]; } /** * get the ids cache length * @return mixed
     */
    public function getArticleIdLen($block) {if(!$this->Cache->exists(ArticleId.$block)) {$this->setIdList($block);
        }
        $count = $this->Cache->llen(ArticleId.$block);
        return $count; } /** * get the ID interval cache * @param$start
     * @param $end
     * @return mixed
     */
    public function getRangeId($start.$end.$block) {if(!$this->Cache->exists(ArticleId.$block)) {$this->setIdList($block);
        }
        $cache_ids = $this->Cache->lrange(ArticleId.$block.$start.$end);
        return $cache_ids; } /** * Set the ID list cache * @return mixed
     */
    public function setIdList($block) {$lists = M('Articles') - >where("DATE_FORMAT(created_time,'%Y-%m') = '".$block."'")->order('id ASC')->select();
        $ids = [];
        if(count($lists)) {$ids = array_column($lists.'id');
            foreach($ids as $value) {$this->Cache->rpush(ArticleId.$block.$value); }}} /** * Get a record * @param$id
     * @return mixed
     */
    public function getInfo($id) {$info = $this->Cache->get(ArticleList.$id);
        if(!$info){// a cache does not exist$info = M('Articles') - >where('id = '.$id)->find();
            $this->Cache->set(ArticleList.$id.$info, 3600); // Record missing cache}return $info; } /** * Add by id * @param$data
     * @return mixed
     */
    public function create($data) {$id = M('Articles')->add($data);
        $this->Cache->rpush(ArticleId.substr($data['created_time'], 0, 7),$id); // Insert the end of the listreturn $id; } /** * Update by id * @param$data
     * @param $id
     * @return mixed
     */
    public function updateById($data.$id) {$res = M('Articles') - >where('id = '.$id)->save($data);
        $this->kellActicleInfoCache($id);
        return $res; } /** * delete by id * @param$id
     * @return mixed
     */
    public function deleteById($id) {$data = M('Articles')->field('created_time') - >where('id = '.$id)->find();
        $res = M('Articles') - >where('id = '.$id)->delete();
        $this->kellActicleInfoCache($id);
        $this->Cache->lrem(ArticleId.substr($data['created_time'], 0, 7),$id); // Delete the ID list cachereturn $res; } /** * remove the content cache * @param$id
     */
    public function kellActicleInfoCache($id) {$this->Cache->rm(ArticleList.$id); // Delete a cache}}Copy the code

3)zset

Redis paging cache method implementation

1. Set the cache directly. If there is a large amount of data, add, delete, or modify operations, the cache update frequency is high and the efficiency is low.

2. Set the cache by page number.

3. One cache for one data

Questions raised:

Question 2:

Solution:

Related Posts

In-depth analysis: annotations and reflection, annotation custom and reflection specific use examples to help you understand

Share the React component across multiple applications

Sa-token permission authentication