Let’s get into Dachang series -Redis dual write consistency, concurrent contention, threading model

omg

Boys why not take wu hook, charge guanshan fifty states FPX 🐂B, LPL two consecutive crown 🏆 🐂B!

Looking at the golden rain falling, I went to the window, found the sky a little blue, the wind a little cotton, my canthus and wet!

Recently double eleven is a little busy to say the truth, direct liver explosion, is so as a warm male I, or give you squeeze out time to make the final chapter, can not help but give their praise 👍

Put a double eleven photo to prove really busy, hope not to take off!!

I should still be sleeping while you’re watching lol. Trapped 🛌

What I told you before, limiting traffic and downscaling, did it come true again on Singles’ Day? Actually, the ordering interface was not suspended. Sacrificing part of user experience and saving the server, you can succeed with a few more clicks.

Last year the refund interface was broken, and this year Ali is clearly smarter.

The body of the

We’ve covered a lot about Redis in the last series. If you haven’t seen it yet, please review it

• Punch the Interviewer series -Redis Basics
• Punch the Interviewer series – Cache avalanche, Breakdown, penetration
• “Punch the Interviewer” series -Redis Sentry, Persistence, Master-slave, hand rip LRU

That mentioned Redis and I’m sure you’ll have to deal with the basic types of usage scenarios in the interview or in the actual development process, problems with concurrency competition, and problems with double-write consistency in the cache database, and so on. Let’s welcome our next victim.

The interview began

A paunchy middle-aged man in a plaid shirt walks up to you with a scratched MAC, looks at his bald hair and thinks he’s a top architect. But we have poems and books in our stomach. (Isn’t that the interviewer for the first article?)

Young man, do you remember in chapter 1 I asked you how many basic data types Redis has?

HMMMM, handsome interviewer, I must remember, no teeth unforgettable!!

Thank you so much for not giving me an Offer!

Can you tell us about their features and different usage scenarios?

Okay, so LET me start with String.

String:

This is the simplest type, just plain set and get, simple KV cache.

However, in the real development environment, many people may convert many complex structures into strings for storage. For example, some people like to convert objects or lists into JSONString for storage, and then reverse sequence.

I won’t discuss the right or wrong here, but I still hope that we can use the most appropriate in the most appropriate scene data structure, the object can’t find the most suitable type of but you can choose the most appropriate, after a look at the specification, so someone else to take over your code! This guy is a little things ah, see you what is done with String, rubbish!

Well, these are all digressions, the truth or hope we keep in mind, habits become natural, small habits make you.

The actual application scenarios of String are as follows:

• Cache function: String String is the most commonly used data type, not only Redis, each language is the most basic type, therefore, using Redis as a cache, with other databases as a storage layer, using Redis support high concurrency characteristics, can greatly speed up the system read and write speed, and reduce the back-end database pressure.

• Counter: Many systems will use Redis as a real-time counter of the system, which can quickly realize the function of counting and querying. And the final data results can be dropped to the database or other storage media for permanent storage at a specific time.

• Shared user Session: When users refresh the interface again, they may need to access the data for re-login or access the page cache Cookie. However, Redis can be used to centrally manage users’ sessions. In this mode, only the high availability of Redis is required, and each update and acquisition of user sessions can be completed quickly. Greatly improve efficiency.

Hash:

This is a Map like structure, which generally allows you to cache structured data, such as an object (provided that the object is not nested with other objects) in Redis, and then operate on a Hash field every time you read or write from the cache.

But this scenario is actually a little bit more simple, because many objects are more complex now, for example, your commodity object may contain a lot of properties, there are also objects. I don’t use my own scenes that much.

The List:

List is an ordered List, and you can do a lot of things with this.

For example, you can store a List of data structures, such as a List of fans, a List of comments on articles, and so on.

For example, you can read elements in a closed interval through the lrange command, and you can achieve paging queries based on List. This is a great function. Based on Redis, you can achieve simple high-performance paging, which can do things like microblog that drop down constantly paging, with high performance, and go page by page.

For example, you could make a simple queue of messages, and then you could pop them in from the head of the List and pull them out from the bottom of the List.

The List itself is one of the most common data structures we use during development, not to mention hot data.

• Message queue: Redis linked list structure, can easily achieve blocking queue, can use left in right out of the command composition to complete the queue design. For example, producers of data can insert data from the left using Lpush, and multiple consumers can “grab” data at the end of the list using BRpop blocking.

• Article list or data page display applications.

  For example, we often use the blog site list of articles, when users more and more large, and each user has its own list of articles, and when the article, all need paging display, then can consider to use Redis list, list not only orderly access elements is also supported in accordance with the scope, can perfectly solve the paging query function. Greatly improve query efficiency.

Set:

A Set is an unordered Set, automatically unduplicated.

If you need to do a quick global de-weighting of some data, you can also do it based on a HashSet in the JVM memory, but what if your system is deployed on multiple machines? Redis is used for global Set de-duplication.

We can use Set to perform intersection, union, and difference operations. For example, we can Set the intersection of two people’s friend lists and see who their common friends are. Isn’t it.

Anyway, these scenes are more, because the comparison is quick, the operation is also simple, two queries a Set to fix.

Sorted Set:

A Sorted set is a Sorted set. It is unrepeatable but Sorted. It is given a score when it is written in.

The usage scenario of ordered sets is similar to collections, but set sets are not automatically ordered. In contrast, Sorted sets can use scores to sort members, and they are Sorted when they are inserted. So when you need an ordered and non-repetitive list of collections, you can choose the Sorted set data structure.

• Leaderboards: An ordered collection of classic usage scenarios. For example, a video website needs to make a list of videos uploaded by users. The maintenance of the list may be in many aspects: by time, by play quantity, by the number of likes obtained, etc.

• Sorted Sets are used to create weighted queues, such as a score of 1 for common messages and 2 for important messages, and then the worker thread can choose to obtain work tasks in reverse order of score. Prioritize important tasks.

  Weibo hot search list, is a behind the heat value, the front is the name

  

summary

There are five basic types of Redis, which I have also mentioned in the basic. In fact, this question is generally for those who are below P6, that is, 1-3 years old or so, they may ask more questions.

I think you can answer the five types, but I don’t know if you know the specific use of the five types, and when to use which type is the most appropriate?

If you don’t give a good answer, including a few data types and a few scenarios, then you’re done. You’re not going to impress the interviewer who thinks you just do simple sets and gets. So seemingly very simple interview questions are actually the most easy to see your depth, we should pay attention to lay a good foundation.

Have you ever considered the data issues that Redis presents if you have multiple systems operating at the same time?

Yes, YES, THIS is a problem I have encountered in the past in development, but it does happen in concurrent processes, such as the following situation

System A, B and C operated the same Key of Redis respectively. It was normal that the order was 1, 2 and 3, but because the network of system A suddenly jutted, B and C operated Redis in front of it, so the data was wrong.

Just like order, pay, refund three order you change, you refund first, then order, then pay, the process will fail, the data will not be chaotic? You paid for a refund before your order was generated? It’s obviously not going anywhere. It’s a horrible thing to do online.

So how do we solve this situation?

We could have hired a butler to help us manage our data.

At some point, multiple system instances are updating a key. Distributed locking can be implemented based on Zookeeper. Each system obtains distributed locks through Zookeeper to ensure that only one system instance can operate a Key at a time, and no one else can read or write the Key.

You to write to the cache data, has been found out from the MySQL, have written to MySQL, the writing time must be saved in the MySQL a timestamp, from MySQL found out, the timestamp is found out.

Before writing, check whether the timestamp of the current Value is newer than the timestamp of the Value in the cache. If so, you can write; otherwise, you cannot overwrite new data with old data.

As long as you use cache, you may involve the cache and database double storage double write, as long as you are double write, there will be data consistency problem, so how do you solve the consistency problem?

In general, it is best not to serialize read and write requests into an in-memory queue if the cache is allowed to slightly inconsistencies with the database, that is, if your system is not strictly “cache + database” consistent.

Serialization guarantees that no inconsistencies will occur, but it can also result in a significant throughput reduction of the system, requiring several times more machines than normal to support a single request on the line.

Put some columns of operations into the queue, the order will not be out of order, but the high concurrency, the queue is easy to block, but will become the weakness of the whole system, bottleneck

Do you know the most classical KV, DB read and write mode?

The most classic Cache + database read/write Pattern is Cache Aside Pattern

• If the cache does not exist, read the database, and then fetch the data and put it in the cache, and return the response.
• When updating, update the database first and then delete the cache.

Why delete the cache instead of update it?

The reason is simple. A lot of times, in a more complex caching scenario, the cache is not just a value pulled straight out of the database.

For example, one field of a table may be updated, and the corresponding cache may need to query the data of the other two tables and perform operations to calculate the latest value of the cache.

Also, updating the cache can be costly. Does that mean that every time you change a database, you have to update the corresponding cache? This may be true in some scenarios, but not in more complex scenarios where cached data is computed. If you frequently modify multiple tables involved in a cache, the cache updates frequently. But the question is, will this cache be accessed frequently?

For example, if a table’s fields are changed 20 times in a minute, or 100 times, the cache is updated 20 times, or 100 times. But the cache was only read once in a minute, and there was a lot of cold data.

In fact, if you just delete the cache, the cache is recalculated within a minute, and the overhead is significantly reduced. Cache is what you need to cache.

In fact, deleting the cache, rather than updating it, is the idea of Lazy computing. Instead of redoing a complex calculation every time, whether it’s needed or not, let it recalculate until it needs to be used.

Like Mybatis, Hibernate, have lazy loading idea. Query a department, the department carries a List of employees, there is no need to say that every time you query a department, the data of 1000 employees in the department will be found at the same time. 80% of the time, the department, you just want to access the information in that department. Search the department first and access the employees in the database at the same time, so only when you want to access the employees in the database, you will query 1000 employees in the database.

What is the difference between Redis and Memcached, and why did you choose Redis as your cache middleware?

Redis supports complex data structures:

Redis has more data structures than Memcached and can support richer data operations. If you need a cache that can support more complex structures and operations, Redis is a good choice.

Redis native supports cluster mode:

In redis3.x, Cluster mode is supported. Memcached has no native Cluster mode and relies on the client to write data in fragments to the Cluster.

Performance comparison:

Because Redis uses only one core, while Memcached can use multiple cores, Redis performs better than Memcached at storing small data on average per core. For data over 100K, Memcached performed better than Redis, which has recently been optimised for storing big data, but not as well as Remcached.

Do you know why you use this stack? Why do you choose this technology stack, have you done the comparison of technology selection, do you understand the advantages and disadvantages, you do not know anything, just for the purpose of using, then you may almost mean.

Redis thread model?

Redis uses the file event handler internally. This file event handler is single-threaded, so Redis is called the single-threaded model. It uses IO multiplexing mechanism to monitor multiple sockets at the same time, and selects the corresponding event processor for processing according to the events on the Socket.

The structure of the file event handler consists of four parts:

• Multiple Socket
• IO multiplexing procedures
• File event dispatcher
• Event handler (connection reply handler, command request handler, command reply handler)

Multiple sockets may concurrently produce different operations, each corresponding to a different file event, but IO multiplexers listen for multiple sockets, queue events generated by the Socket, and the event dispatcher fetches events from the queue one at a time. The event is passed to the corresponding event handler for processing.

The end of the interview

The young man has interviewed you for four times. You speak with reason and clear logic, and you must be a good player after coming to the company. I want you to be my Leader, haha?

The interviewer don’t joke with me, I still have a lot of gap with you like accumulated technical experts, your experience and technical depth, without a long time of hone is unable to reach, I still have to learn more from you.

Ok, young man, you have something, you are young and ambitious, you don’t feel inferior, you know what is precious, it is you come to work.

Good interviewer, but I think I have a lot of knowledge in Java basics, MQ, Dubbo, etc. You haven’t asked me, why don’t you meet me next time?

Forced, for the next phase of ambush pen ha ha, the next period to write what you set!!

You can’t help but give yourself a thumbs-up for making it to the end!

(Hint like, every time read not like, you want to whoring me? You guys are good, but I like it).

---

The end of the Redis series —-

---

conclusion

Since all said is the final chapter of Redis, I finally do a Redis often met questions, summary of the topic, the answer you have to think about my previous article basically mentioned, the results can go to my public account reply [answer] to obtain, but I still hope that you can see the topic can think of the answer, and remember in mind, Teaching everyone how to answer is just to help everyone organize the language, the real scenario solution is to let everyone understand.

(The answer will come after Wednesday. I will sleep for a while.)

• How are Redis keys addressed in clustered mode? What are the algorithms for distributed addressing? Do you know consistent Hash algorithms?
• 1. What are the benefits of Redis?
• What are the advantages of Redis over Memcached?
• 3, Redis common performance problems and solutions
• MySQL has 2000W data, Redis only 20W data, how to ensure that the data in Redis is hot data?
• 5. What are the differences between Memcache and Redis?
• 6. What are the common performance problems of Redis? How to solve it?
• 7. In what scenarios can we make full use of the features of Redis and greatly improve the efficiency of Redis?
• 8, Redis cache avalanche, penetration, breakdown understand? What are the similarities and differences? How to solve them respectively?
• 9. What are the basic types of Redis? Do you understand their usage scenarios? Have you used more advanced language?
• 10. How does Redis master and slave synchronize data? How to ensure high availability of the cluster? Do you understand persistence?
• 11. Why can Redis single thread support high concurrency?
• 12. How to ensure the consistency of cache and database data?
• 13. How is the cache used in the project? What are the problems caused by the use of cache?

Ramble about +

Finally, I want to say that these four chapters only introduce some common questions in Redis interview. In fact, there are many points THAT I have not answered. It may be enough for us to deal with the interview, but we technicians still need to keep awe of the technology.

You will always only use, do not consider the problems that will be brought by using, and the solution after the problem, I think you will most likely stagnate, since you have entered this line, why not arm yourself.

Learning technology was in fact a process of feedback, learn knowledge breadth, depth, maybe you’re just feel up, a knowledge point in this way, you two, three points you such, eventually you find that your technology is the same as side P6 of different, so that you may be in the team the contribution of major projects are up, the chances of promotion the P7 is big, Money is not up, girlfriend is not beautiful, the house is not big.