preface
We talked about the basic principle and use of Bloom filter two days ago. Well, brother’s gonna have to do it again, because I’m your brother. (Silently moved is good)
This article focuses on the use of Bloom filters, such as cache avalanche, video push and other cases to explain. If you are not familiar with the bloem filter principle, you can go to the history article and read the previous article.
Redis cache penetration solution case
In essence, bloom filters act as a blacklist or whitelist. We analyze cache penetration from these two perspectives.
Whitelists resolve cache penetration
Pay attention to the problem
- If data that is not whitelisted is misjudged to be in the filter, it will pass through to the database, but the probability of miscalculation is very small, so penetration is not a problem.
- All query keys must be placed in bloom filters and Redis, otherwise the request will be returned with empty data.
Code implementation
Elder brother according to this flow chart to give you a code, other several cases, we think how to achieve.
pom
<dependency>
<groupId>org.redisson</groupId>
<artifactId>redisson-spring-boot-starter</artifactId>
<version>3.134.</version>
</dependency>
Copy the codeJava code
import com.alibaba.fastjson.JSON;
import com.bilibili.itlaoge.model.User;
import org.redisson.Redisson;
import org.redisson.api.RBloomFilter;
import org.redisson.api.RBucket;
import org.redisson.api.RedissonClient;
import org.redisson.config.Config;
/** * Resolve cache penetration - whitelist */
public class RedissonBloomFilter {
/** * Redisson */
static RedissonClient redisson = null;
static RBloomFilter<String> bloomFilter = null;
static {
Config config = new Config();
config.useSingleServer().setAddress("Redis: / / 127.0.0.1:6379");
/ / Redisson construction
redisson = Redisson.create(config);
// Build a Bloom filter
bloomFilter = redisson.getBloomFilter("userIdFilter");
// Put the query data into Redis cache and Bloom filter
initData(redisson, bloomFilter);
}
private static void initData(RedissonClient redisson, RBloomFilter<String> bloomFilter) {
// Initialize the Bloom filter: the expected element is 100000000L, with an error rate of 3%
bloomFilter.tryInit(100000000L,0.01);
// Insert data with id 1 into the Bloom filter
bloomFilter.add("1");
bloomFilter.add("2");
// Insert user data with id 1 into Redis cache
redisson.getBucket("1").set({id:1, userName:' zhang3 ', age:18}");
}
public static void main(String[] args) {
User user = getUserById(2L);
System.out.println("User object is:" + JSON.toJSONString(user));
}
public static User getUserById(Long id) {
if (null == id) {
return null;
}
String idKey = id.toString();
// Start simulating cache penetration
// Front-end query request key
if (bloomFilter.contains(idKey)) {
// If the filter is whitelisted, go to Redis to query the real data
RBucket<Object> bucket = redisson.getBucket(idKey);
Object object = bucket.get();
// If Redis has data, return that data directly
if(null ! = object) { System.out.println("Got it from Redis.");
String userStr = object.toString();
return JSON.parseObject(userStr, User.class);
}
// If Redis is empty, query the database
User user = selectByDb(idKey);
if (null == user) {
return null;
} else {
// Flush the data back into the cache
redisson.getBucket(id.toString()).set(JSON.toJSONString(user));
}
return user;
}
return null;
}
private static User selectByDb(String id) {
System.out.println("Query from MySQL");
User user = new User();
user.setId(1L);
user.setUserName("Zhang");
user.setAge(18);
returnuser; }}Copy the codeThe user object
/** * user entity class * @author HP */
public class User implements Serializable {
public static String maYunPhone = "18890019390";
private Long id;
/** * User name */
private String userName;
/** * age */
private Integer age;
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public Integer getAge() {
returnage; } public void setAge(Integer age) { this.age = age; }}Copy the codeBlacklist addresses cache penetration
Pay attention to the problem
- Data in bloom filter is misjudged. If normal data is misjudged to be in the blacklist, empty data will be returned directly.
- The data in the blacklist should be very comprehensive, otherwise there will be more serious penetration problems.
- The illegal data in the blacklist at first may be normal data later. For example: use the number id greater than 1 million to request, we only have 100,000 data in the database, at this time if the ID into the blacklist. When you get to a million, there’s a problem.
To consider
If the blacklist mode and the white list mode combined, do you know how to use? I want you to think about it a little bit.
Examples of application scenarios
The following scenarios are only examples and do not involve service implementation. The purpose is to better understand the use of bloom filters as blacklists and whitelists.
Video Push Scenario (Blacklist)
Background: A video website pushes videos to users
Bloom filter function: used when blacklist.
Requirement: For a user, the video that has been pushed will not be pushed.
Process: When pushing a batch of videos to the user, judge whether these videos exist in the filter first; If it exists, it will not be pushed to the user, and if it does not exist, it will be pushed to the user; At the same time, the pushed video will be stored in the filter blacklist to prevent repeated push.
Cases of reposting videos/articles (whitelist)
Background: a user wants to reprint my elder brother’s article.
Bloom filter function: when whitelist is used.
Requirements: in the elder brother forward white list, have the authority to forward the article.
Process: some user wants to forward elder brother’s article, because did not in white list, forward failure. Then find elder brother open whitelist, elder brother joined him in the whitelist, allow forwarding.
conclusion
No technology is perfect, and every technology has strengths and weaknesses. One technology is not suitable for all businesses, what we need to do is to make use of the advantages of technology to achieve our business needs.
Just like each of us has our strengths and weaknesses, we should explore our strengths and do what suits us.
It is like asking me to be the CEO, and the CEO will write code for me. Although it is very cool, it is estimated that in a short time, the company will be yellow.
