Thank you for your support during this period of time, pay attention to my wechat: South Orange RYC, reply to Yun Xiaoxiao, you can get the latest benefits of a beautiful photo, what are waiting for, come to join us soon ~
“Tomorrow is the final, I only know how to use magic tools without practicing spells.” .
In the dugout room, a jealous Chen Yanyu stared at Li Xiaogeng, who was wiping (showing off) the instrument, teased.
Chen Yanyu, Tongliao State Khan city people, three years ago to break through the foundation period, on the way to the tiandao Sword was abducted by the murderer, when waiting for the wake up has appeared in the ten thousand law fairy door. A man whose voice sounded exactly like that of his kidnapper was standing beside him. He claimed to be the director of the Teaching and Research Section of the Gate. According to the description of the man was fighting chivalry, Chen Yanyu was found to be evil tiandao Sword of the evil repair seized, then anger from the heart, with the war three hundred rounds, finally rescued himself, and in order to compensate for their own forced income under the door.
“Do you know what is the biggest difference between people and animals?” Li Xiaogeng ignored the future of his defeated provocation.
“You know what scares you the most about being a database type compiler?” Chen Yanyu see Li Xiaogeng so defiant, no longer stay in the lounge, turned toward the arena: “I hope you don’t go back to be spanked by your master.”
“Well, it’s not impossible.”
“?”
The final scene, because it is the foundation period competition, so the audience is not much, mainly divided into three parts: Li Xiaogeng fan sister and a small number of fan younger brother, before the Defeat of Chen Yanyu and Li Xiaogeng, and the level is not enough to watch the jiedan period competition lian Qi period disciple.
“Brother Li Xiaogeng is so handsome! Brother Li Xiaogeng will win!”
“Hum, if IT hadn’t been for my poor move, I would have been one of the two people on the stage today.”
“The explanation? Get the commentary in place, it’s about to begin!”
As the traditional old strong school, wan Faxianmenmenjie big than the basic order is still some, at least there will not be any selling melon seeds peanuts submarine floating vendors.
On the ring.
Li Xiaogeng is still the one in white (exclusive school uniform of Cloud Temple), with the red and white flying sword like feathers looming behind, like an angel, a glance and a smile are all picture scroll.
Chen Yanyu there is a lot of ordinary, ordinary ten thousand method immortal door disciples clothing plus ordinary hand small box.
Looking at the challenge under the largest organization “Li Xiaogeng supporters”, Chen Yanyu secretly thought: “Thirty years of Hedong, thirty years of Hexi, weekdays only teach you Li Xiaogeng arson, today I Chen Yanyu also want to light!”
“Li Xiaogeng, do you know JMeter?” Chen Yanyu raised the hand of the small box, just like the silver horn king with a gourd.
“What if you know, what if you don’t?” Li Xiaogeng skimmed the dusty box and looked at the angel wings behind him, laughing.
“Good!” Chen Yanyu was shocked by Li Xiaogeng’s arrogance. Instead of being angry, he laughed, “I will tell you what JMeter is today.”
JMeter suddenly floated in the air, while Chen Yanyu waved his hands as if playing the piano. Numerous electromagnetic waves, which only xiuzhen personage could see, surged out from the box to Li Xiaogeng.
jmeter -n -t testplan/RedisLock.jmx -l testplan/result/result.txt -e -o testplan/webreport
Copy the code“I don’t know if you know what breakdown, avalanche and penetration are after all the time you’ve been using database classers!” Chen Yanyu seems to have seen his victory.
As a normal zhuji period disciple, to participate in the ordinary school of big than, all the way to defeat the temple excellent disciples finally went to the final, but found that the final opponent is not only handsome and deadly, but also from the beginning took out a lot of golden Dan real people do not have the big kill redis cluster, for anyone is collapsed. However, for this database type of method, Chen Yanyu or had an understanding, only through a large number of data breakdown each other a point, they can let its overload restart. At that time, he and the boy can face off, and even the overloading will affect the other side’s processing ability.
“It seems that you know very little about Redis.” Li Xiaogeng even directly back to Chen Yanyu, behind the red and white small sword composed of wings without wind automatic, completely ignored The attack of Chen Yanyu.
Redis solves cache avalanche, breakdown, and penetration problems
1, 1 Cache avalanche
As the name suggests, you’ve all seen avalanches, both in real life and online. It was kind of an avalanche, and for databases, a cache avalanche is kind of an avalanche.
At the same time, the Redis cache fails in a large area. At that moment, it seems that Redis does not exist. At this time, the data is directly requested to the database. If you think about it, the point of caching is to reduce DB. If the cache is gone, a large number of requests will not directly hit the database.
How does the cache avalanche occur?
- A large number of caches fail at the same time, and may be generated and expire at the same time
- The cache is deleted at the same time
- There is an error in the cache layer and it is not working properly
Solutions:
1. When storing data in batches to Redis, increase the expiration time of each KEY randomly to ensure that it will not fail at the same time
2, set the hotspot data never expire, update the cache if there is an update operation (but this method is not good, never expire will result in a large amount of cache, many cache may not be useful)
3, Redis cluster deployment, the hot data evenly distributed in different Redis libraries can also avoid all failures, to avoid Redis problems caused by cache avalanche
1, 2 cache breakdown
There is a Key is very hot, in a large number of concurrent, concurrent access to this point, when the Key fails the moment, a large number of concurrent breakdown cache, direct access to the database.
In fact, cache breakdown is really not a big problem, after all, not every company has such a big hot spot on the same Key, just need to set the expiration time, stable Redis cluster, cache breakdown is not difficult to avoid.
Solutions:
1. Set the hotspot data to never expire
2, ADD mutex (simply put, not load db immediately when the cache fails, but set a mutex key using some of the cache tools with the return value of the successful operation (such as Redis SETNX or Memcache ADD). When the operation returns a success, load db and set the cache.
In my experience, mutex is obviously not necessary. Why use a lock when a hot spot never expires? Isn’t that a gratuitous increase in complexity? Maybe you can see it in special situations, but for me, I can only see it in your blogs.
1. 3 Cache penetration
In name, cache penetration is similar to cache penetration. In fact, pages are more similar, but there are some differences now that they are separated
The concept of cache penetration is very simple. The user wants to query a data and finds that the Redis in-memory database is not there, that is, the cache is not hit, so they query the persistence layer database. Found no, so this query failed. When there are many users, none of the caches are hit, so they all request the persistence layer database. This puts a lot of strain on the persistence layer database, which is equivalent to cache penetration.
But what cache penetration really protects against is hackers.
If a hacker deliberately queries data that is not necessarily in the cache every time, causing each request to go to the storage tier to query, the cache would be meaningless. If the database may fail under heavy traffic, this is also a cache breakdown. Solutions:
- 1. Add parameter verification
- 2. Add a configuration item from the gateway layer Nginx to mask the number of accesses per second from a single IP address beyond the threshold
- 3. Bloom Filter can prevent cache penetration well. Its principle is also very simple, which is to use efficient data structure and algorithm to quickly determine whether your Key is in the database
Redis uses bloom filters to prevent cache breakdowns, mainly by putting existing caches into Bloom filters. When hackers access non-existing caches, they quickly return to avoid cache and DB failure.
1, 4 Bloom filter
Essentially, The Bloom filter is a kind of data structure, a clever probabilistic data structure, which is characterized by efficient insertion and query and can be used to tell you “something definitely doesn’t exist or might exist”.
A Bloom filter consists of a long array of bits and a series of hash functions.
Each element of the array takes up only 1bit of space, and each element can only be 0 or 1.
The Bloom filter has k hash functions. When an element is added to the Bloom filter, k hash functions are used to evaluate it k times to get k hash values. Based on the hash values obtained, the value of the corresponding subscript is set to 1 in the bit array.
To determine whether a number is in the Bloom filter, the element is hashed k times, and the resulting value determines whether each element in the array is 1. If each element is 1, the value is in the Bloom filter.
As more and more elements are inserted, when an element that is not in the Bloom filter is hashed with the same rules, the resulting values are queried in the array, perhaps because the other elements were set to 1 first.
So the Bloom filter can misjudge, but if the bloom filter determines that an element is not in the Bloom filter, then the value must not be in the Bloom filter.
Let’s go further
1, 5 cuckoo filter
In order to solve the problem that the Bloom Filter cannot delete elements, the author of Cuckoo Filter: Better Than Bloom proposed a Cuckoo Filter. Compared to bloom filters, cuckoo filters have the following advantages: better query performance, more efficient space utilization, support for reverse operation (delete), and counting.
The cuckoo filter takes its name from the fact that it adopts the same parenting method as the cuckoo. The original cuckoo hashing method uses two hash functions to hash a key to get two positions in the bucket.
- If both positions are empty
keyRandom deposit into one of these locations- If only one location is empty, save to the empty location
- If none is empty, a random element is kicked out, and the element is then recalculated to find its position
The cuckoo filter is derived from the cuckoo hash algorithm, which cleverly designs a unique hash function so that p2 can be computed directly from p1 and element fingerprints, without the need for the full x element.
And as you can see from the following formula, when we know fp and P1, we can immediately figure out p2. And likewise, if we know p2 and FP, we can just figure out p1.
fp = fingerprint(x) p1 = hash(x) p2 = p1 ^ hash(fp) / / or Copy the codeSo we don’t need to know whether the current position is P1 or P2 at all, we just need to xor the current position and hash(FP) to get the dual position. And just make sure to hash(FP)! Lambda equals 0, so p1 factorial is guaranteed. = p2, so you don’t have to kick yourself and cause an endless loop.
After listening to Li Xiaogeng’s introduction, Chen Yanyu suddenly laughed and said, “I see. I thought my JMeter could control the database, so the big data attack could penetrate your database, but I never thought it was me.”
“In fact, you don’t have to be upset, your idea is right, the best weapon to control the database is a large amount of data, but to penetrate my entire Redis cluster, your single server running JMeter is not enough to see.” Chen Yanyu comfort way, as a winner naturally should have a winner’s decent.
“Even if I have enough JMeter servers, you must have a back-up.” Chen Yanyu asked unwillingly.
“Bingo!” Li Xiaogeng step by step close to the Redis cluster locked breath of Chen Yanyu, gently remove the lost master magic support can only quietly floating in the air of the box: “since Redis can store data, nature can also real-time delete redundant data.”
Second, memory elimination mechanism
Since Redis can store data, it naturally needs to be able to delete excess data, otherwise, the space is full, where to put the new content? Smart programmers have come up with two solutions to Redis’s memory problems.
- 1. Scheduled deletion: Delete expired data through a timer
- 2, lazy delete – query if expired do not return, expired delete. This situation tends to have a lot of redundant data that takes up a lot of space
2. 1. Delete periodically
Create a timer. When the key is set to expire and the expiration time reaches, the timer task will immediately delete the key. By default, the timer task will randomly select some keys to determine whether they are expired or not.
Advantages: saving memory, then delete, quickly release unnecessary memory occupation
Disadvantages: The CPU is under great pressure. No matter how high the CPU load is at this time, it will occupy THE CPU, which will affect the redis server response time and instruction throughput
2. 2
When data in Redis reaches its expiration date, we do not process it. If the data has not expired, it will return normally. If it is found that the data has expired, delete it immediately, and return that it does not exist, and exchange storage space for processor performance (exchange space for time).
Advantages: Saves CPU performance and deletes the vm only when the vm must be deleted
Disadvantages: High memory pressure, long – term memory occupying data
I don’t know if you’ve noticed, but most algorithms are either time for space or space for time. This is one of the ultimate mysteries of humanity, and if we only knew it, we could solve most of our problems.
2, 3 elimination mechanism
The flushing mechanism can also be set by searching for maxmemory-policy in the Redis redis.config file
Noeviction: Any commands that cause memory allocation will bug when memory usage reaches its threshold. Allkeys-lru: In the primary key space, the recently unused keys are preferentially removed. Volatile -lru: Removes the most recently unused key from the key space with expiration time. Allkeys-random: Randomly removes a key from the primary key space. Volatile -random: Randomly removes a key from an expired key space. Volatile - TTL: In a key space with an expiration time set, the key with an earlier expiration time is removed first.Copy the code
“You’ve practiced the thread pool magic, so you know that a proper flush strategy can solve a lot of problems, and with Redis, I just need to change the parameters properly so that the most recently stored data can be flushed quickly to prevent excessive memory stress.” Li Xiaogeng snapped his fingers to release the redis cluster’s lock on Chen Yanyu, and put JMeter into his hands: “In the final analysis, the hardware gap is too big, no matter how good the strategy is, it is useless.”
“What are you going to do? Chen Yanyu looked at Li Xiaogeng’s behavior in bewilderment: “Are you humiliating me?”
“Aren’t you going to confront me? I’ll give you the chance!” Then, the Redis cluster, like a swarm of small bees, returned to Li Changgeng’s space ring.
Yen-yu Chen has been impressed by Li Xiaogeng intellect and bearing, originally wanted to throw in the towel, but to give each other a good impression, then maximum luck “Java true through”, go all out, “chang gong brother, though I know no matter whether to use multiplier, your capacity for all on me, but you also need to be careful oh, I won’t sent you too much!”
(Note: Those who are higher in the same sect are senior brothers)
Hello, I am the south orange, ten thousand method immortal door of the door, just from the kyushu world through the earth, because of the impact of space-time turbulence lead to my magic all lost, now have to pass this platform to the vast xiuzhen genius to borrow strength. Every “like” and “attention” from you will help me to come back to kyushu world. Watch for WX: Southern Orange RYC.
After I go back, we are all the elders of ten thousand method immortal door, I will give you countless days of material to treasure, everyone such as dragon, the whole people soar.