1. List problems
Select * from sort ASC where id desc; select * from sort ASC where ID desc; What do I do with Redis?
[analysis]
First of all, there may be a problem with this problem itself, because if redis is limited, then the method of dealing with it will be limited. At that time, I was so preoccupied with redis that I forgot to think whether Redis is suitable for dealing with this problem.
Second, let’s talk about how redis handled at the beginning:
1) The goods list is stored in the ordered set, such as the ordered set goods-by-sort:
Product ID (member), value (score)
1 1
2 1
3 3
4 2
2) Whenever a new item is created and added to the queue, the sort value is the default value;
3) Whenever the operator modifies the SORT value in the management background, the score value of corresponding goods in this ordered set will be modified;
4) Delete the item, or the setting is not visible, then delete the data of the item ID from the ordered set;
5) When the user gets the list of items, it needs to be sorted by sort and ID, so I add a new ordered set: Select * from ‘goods_list_data’, rename ‘goods_list_data’, and rename ‘goods_list_data’, and rename ‘goods_list_data’, and rename ‘goods_list_data’, and rename ‘goods_list_data’, and rename ‘goods_list_data’, and rename ‘goods_list_data’, and rename ‘goods_list_data’.
6) Adjustments to the goods_by_sort ordered collection, such as modifications, additions, and deletions, may be made to the goods_BY_sort ordered collection, so the goods_list_data data also needs to be updated, or the user will keep seeing the deleted items. So in these three cases, I’m going to update a string of refresh_goods_data key, incr each time. Check the value every minute with a scheduled task. If it is not 0, update goods_list_data and set refresh_GOOds_data to 0. Otherwise, it is not processed because the ordered goods_list_data collection is unchanged.
[The problem with doing so above]
1) The coupling is very deep and difficult to maintain; I have 2 ordered sets, and I have timed tasks;
2) There will be bottomless problem: more and more data will be stored in the ordered collection. Of course, this can be handled according to business, such as cutting, but maintaining the ordered collection is also a problem.
[Better treatment]
Id desc = redis; id desc = redis; id desc = redis; id desc = redis; This is evaluated based on the amount of data and the complexity of SQL statements. If you are joining tables, or if you have been told that your database is experiencing slow query SQL, you should not use this method.
2) My colleague suggested using Sphinx to deal with this, which makes sense, but I haven’t tried it yet.
2, Redis anti-avalanche, anti-penetration, bottomless hole problems
【 Analysis and solutions 】
Effective ways to prevent avalanche problems:
1) No expiration time is set. As long as the data is updated to Redis in real time, the data given to the user is real time and does not affect the back-end database;
2) Distinguish hot data from cold data, refresh hot data regularly every day;
Anti-penetration problem:
1) Because there is no data in Redis, it will go to the database to read, but the database also does not have, so it will not write back to the cache, resulting in the concurrent access to the database every time round read;
Write a null or false to redis and set an expiration time, such as 2 minutes.
3) It should be noted that this type of key can not be stored for a long time, and the amount of key will occupy more memory.
Bottomless problem:
1) If the key does not expire, it will always be stored in the memory, and the memory will be increasingly insufficient.
2) If the key expires, how to deal with the expiration, high concurrent access data, the database will hang? Internet use setnx many code lock mode, access to the lock to the db queries then write back to redis, and other request didn’t get the lock, then wait for a period of time (such as 10 milliseconds) and then to read redis, take to return, can take less than data according to the business look do you want to directly returns an empty result, or again to get the lock, Until data is read or the number of attempts reaches a specified number.
3. Flush cache problems
For example: Because it is on an old project to do optimization, there is no do before redis cache (strictly speaking, there are still a cache, but it is the result of the API interface cached set an expiration time), so the new optimization after launch, if access to the old data, cache, so if you don’t brush data, will hit all requests to the cache is empty, You can either return no data (very bad user experience), or you can go to the database, which is like an avalanche, and access the database concurrently, and the database may hang, so the better solution is to flush the data to the cache first. What do I do here?
[analysis]
1) Analyze which data needs to be refreshed to the cache in advance based on the business;
2) Add lock mechanism, if can not obtain data, first to obtain the lock, obtain the lock, then go to db query and write back to Redis, db no data write null or false to Redis and set the expiration time;
3) According to the amount of data and services in the database table, analyze whether some data can be brushed first. For example, if there are 10 million pieces of data for a commodity, users can slowly see 10 million pieces of data on the homepage of the website, but most users may only see dozens of pages of data in front of them, such as 20 pieces of data on a page, then prepare 2000 pieces of latest data or popular data. Other goods, waiting for users to access, from Redis, not read from db to write back to Redis.
4. Products are associated according to labels, such as
Item A: Tag1, tag2, tag3
Item B: Tag2, tag4
Item C: Tag1, tag5
D: Tag4
So, good A is related to good B and good C; Commodity B is related to commodity D; Commodity C is associated with commodity A; Good D is associated with good B.
Whenever we want to obtain the goods associated with commodity A, of course, we can obtain the label of A from DB, and then calculate the managed goods, so how to use Redis to deal with it?
[analysis]
If redis is used, it is necessary to calculate the relationship in advance and store it in the set/ordered set. Then, once the data needs to be obtained, there is no need to calculate, but directly read these associated ids from the cache.
1) When new products are added, labels will be attached. Computing associated goods is a time-consuming operation. It can be placed in a queue for periodic processing by background scripts.
2) When deleting goods, it is also necessary to put them into queues for processing by background scripts;
3) Label modification (adding one or more labels to a product, deleting one or more labels, and modifying some labels) also needs to be put into a queue for processing by background scripts;
[The problem with doing so above]
1) It is difficult to maintain; Add, delete, change, all need to maintain this ordered set. The advantage is that the id is directly obtained from the collection/ordered collection when the associated data is needed, and the basic information is also obtained from Redis.
2) I consulted my colleague and said that sphinx could also be used to deal with it, but I haven’t tried it yet.
5, can you solve the mysql like problem with Redis
The answer, of course, is no, you have to use full-text search systems like Sphinx or ES.
6. The logic of the product list is to allow the display of this data for items posted by the user (regardless of the approved status) + items posted by other users (only approved status). At that time, redis was used to deal with two ordered collections, one is to store all approved goods on the website; The second ordered collection is to store items posted by the users themselves, regardless of audit status. When you need to get a list of items, merge the two collections and then paginate the data.
[question]
In concurrent cases, merging ordered collections is costly and may cause congestion;
【 Solution 】
1) If possible, directly query mysql.
2) Use Sphinx: Another colleague suggestion.
Conclusion:
1. Redis is not omnipotent and has its own advantages and disadvantages; Redis is not suited for relational businesses like SQL;
2, the performance of Redis is very good, but artificial operation, improper use, may cause blocking;
3, in addition to redis, there are other methods to handle, can not be limited to death. When dealing with a problem, consider not only whether it can be handled, but also whether it is reasonable.