Napoleon said: Victory belongs to him who perseveres to the last.

And it just so happens that we’re here to talk about how to make systems stand up and survive in the face of massive explosions and even massive toppings — caching.

Victory belongs to the most male. — Napoleon Bonaparte, French military and political leader

The directory system

Let’s take a quick look at the directory system for sharing.

Today I will give you an introduction to the use of cache from five aspects, including principles, practice, technology selection and FAQ.

The catalogue is a skeleton of a human body, and the beauty of the cache comes to life only when it is filled with organs, organs and flesh. I’m going to invite you to do this with me.

Let’s go beyond Hello World and talk about caching.

About the cache

What

What is a cache?

Caching is a very common way to improve performance in the real world.

In Java, the so-called cache is to store the frequently called objects in the memory of the program or system, so that the objects can be quickly retrieved from the memory when called again, without creating new and repeated instances.

Doing so can reduce system overhead and improve system efficiency.

Current caching practices fall into two modes:

  • Memory cache: Cached data is stored in the server’s memory space.

    Advantages: Fast speed. Disadvantages: Limited resources.Copy the code

  • File cache: Cached data is stored in the server’s hard disk space.

    Advantages: Large capacity. Disadvantages: Slow speed, especially in large caches.Copy the code

why

Why cache?

The most succinct answer I’ve seen to why cache is used is that it comes from a dream about how quickly and cheaply a socialist society can be built.

But this is a very contradictory statement, as if you are not tall rich handsome still want to marry white rich beautiful, like a dream.

Because more is impossible to fast, good can not save, how to achieve more and fast, good and save?

The answer is cache!

Let’s talk about how to make this dream come true with caching.

First of all, I want to clarify why I thought of doing such a sharing.

In fact, from the first time to use Java integer cache, to learn about CDN proxy cache, from the first time to contact MySQL built-in query cache, to use Redis cache Session, I increasingly find the importance and universality of using cache.

Therefore, I think it is necessary for me to sort out what I have learned and apply it to work and benefit everyone, hence the creation of such a technology sharing.

Before we talk about caching, let’s talk about databases.

In add, delete, change and search, database query occupies more than 80% of the database operations, very frequent disk I/O read operations, will lead to extremely low database performance.

The importance of databases is self-evident:

  • Database is usually the most core part of enterprise application system
  • The amount of data that databases hold is often very large
  • Database queries are often frequent and sometimes complex

We know that for most Web applications, the bottleneck of the entire system is the database.

The reason is simple: other factors in Web applications, such as network bandwidth, load balancing nodes, application servers (including CPU, memory, disk lights, connection count, etc.), and caching, can easily be improved by horizontal scaling (commonly known as adding machines).

As for MySQL, due to the requirement of data consistency, it is impossible to disperse the pressure of writing data to the database by simply adding machines. Although the pressure can be alleviated by pre-caching (Redis, etc.), read-write separation, sub-library sub-table, but compared with the horizontal expansion of other components of the system, it is subject to too many restrictions, and will greatly increase the complexity of the system.

Therefore, database connection and reading and writing should be treasured.

You might think to just use caching, but using caching in large numbers, regardless of scenario, is simply not scientific. We can’t have a hammer in our hand and see everything is a nail.

However, caching is not a panacea. It is not easy to use it carefully. So I took a moment to sort out the implementation of caching and some common issues.

when

Start with a brief overview of the Web request process and the role of the different node caches.

how

Regardless of the code, a simple caching data request flow is shown below for how caching works.

Two of the most critical cache strategies to consider when designing a cache.

– TTL (Time To Live) TTL: a period of Time from the Time point created in the cache To the Time when it expires (access within this period will expire regardless of whether access is made within this period)

  • TTI (Time To Idle) Indicates the Time that a data is removed from the cache before it has been accessed

When WE talk about cache avalanche later, we’ll talk about how catastrophic it can be if you don’t set the cache policy right, and how to avoid it.

Custom cache

How to implement

Given the concepts of caching, is it difficult to implement caching, or rather front-caching of storage?

The answer is: no.

The JVM itself is a very fast cache store, and Java provides thread-safe ConcurrentMap, which makes it very easy to implement a fully customized cache instance.

As you’ll see later, Spring Cache’s default implementation, SimpleCacheManager, also designs its Cache this way.

Here is the simple implementation code, but in 36 lines, the basic operations of storing, updating, reading, and deleting the cache are implemented. This, combined with actual business code, makes it easy to play with caching in the JVM without relying on any third party implementation.

However, I think as a technical person with pursuit, you will never stop here.

So let’s think about, what are the pros and cons of our custom cache implementation?

You’ll have a better understanding of Spring Cache’s principles and benefits than you would have with a custom Cache.

Spring Cache features are listed here, and its principles and usage are described below.

Spring Cache

Spring Cache is an abstraction of the caching functionality provided by Spring: it allows the binding of different caching solutions (Ehcache, Redis, Memcache, Map, and so on), but does not directly provide the caching functionality itself.

It supports caching in an annotated manner, which is very convenient.

The implementation of Spring Cache essentially relies on Spring AOP’s support for aspects.

Knowing how Spring Cache works, you will have a better understanding of the use of Spring Cache annotations.

There are four main annotations used in Spring Cache.

@cacheevict is very important for cache consistency, which I’ll cover later.

Spring also supports custom Cache keys and SpringEL, which are not covered in detail here, but should be referred to the Spring Cache documentation.

Cache treble

Just as well written music requires a singer to sing it.

As mentioned above, Spring Cache is an abstraction of caching. What are some common implementations of caching?

There are three tenors in the world of singing, so if the caching world had a vote, who would they be?

Redis

Redis is a key-value storage system, similar to Memcached.

The difference is that it supports more value types, including string, list, set, zset, sorted set, and hash. These data types all support push/ POP, Add /remove, and intersection union and difference sets.

As with Memcached, the data is cached in memory for efficiency.

The difference is that Redis periodically writes the updated data to disk or the modified operation to the appended record file, and on this basis realizes master-slave synchronization. Redis supports master/slave synchronization. Data can be synchronized from the primary server to any number of secondary servers, which can be the primary server associated with other secondary servers. This allows Redis to perform single-layer tree replication.

Data can be written intentionally or unintentionally. The fully implemented publish/subscribe mechanism makes it possible to subscribe to a channel and receive a complete record of message releases from the master server when the tree is synchronized anywhere from the database.

Synchronization helps with scalability and data redundancy for read operations.

What scenarios are suitable for Redis?

  1. Session Cache: Caching sessions in Redis has an advantage over other stores, such as memcached, because Redis provides persistence.
  2. Full page Caching (FPC) : In addition to the basic session token, Redis also provides a very simple FPC platform.
  3. Queues: One of Redis’s strengths in the memory storage engine world is that it provides list and set operations, which makes Redis a great platform for message queuing.
  4. Leaderboards/counters: Redis does a great job of incrementing and decrement data in memory.
  5. Subscribe/publish

Disadvantages:

  1. Persistence. Redis stores the data directly in memory, and to save the data to disk, Redis can implement the persistence process in two ways.

    Periodic snapshot: Writes the entire database to disk at intervals. All data is written each time, which costs a lot. Statement appending (AOF) : Only trace changed data, but the appending log may be too large, and all operations are re-executed, resulting in slow recovery.

  2. Memory consumption is too high.

Ehcache

Ehcache is a mature caching framework that you can use directly to manage your cache.

EhCache EhCache is a pure Java in-process cache framework. It is fast and clean. It is the default CacheProvider in Hibernate.

Feature: you can configure the memory, enabling the disk cache (maxEntriesLoverflowToDiskocalDisk configuration when memory object number about maxElementsInMemory Ehcache will object to disk).

Memcached

Memcached is a high-performance distributed memory object caching system for dynamic Web applications to reduce database load. It is based on a HashMap that stores key/value pairs.

Its daemons are written in C, but clients can be written in any language and communicate with the daemons over the memcached protocol.

Memcached speeds up a dynamic, database-driven web site by caching data and objects in memory to reduce the number of database reads.

Belonging to the same key-value storage system, Memcached and Redis are often compared together:

  1. Memcached’s data structures and operations are simple and not as rich as those supported by Redis.
  2. Memcached is much more efficient with simple key-value storage than with Redis hash key-value storage because of its combined compression.
  3. Because Redis uses only one core, while Memcached can use multiple cores, Redis performs better than Memcached at storing small data on average per core. For data over 100K, Memcached outperforms Redis, which has recently been optimized for storing large data, but still lags behind Memcached.
  4. Although Redis is a memory-based storage system, it itself supports persistence of in-memory data and provides two main persistence strategies: RDB snapshots and AOF logging. Memcached does not support data persistence. Memcached is a full-memory data caching system. Redis supports data persistence, but full memory is the essence of its high performance.
  5. As a memory-based storage system, the size of the physical memory of the machine is the maximum amount of data that the system can hold. If the amount of data that needs to be processed exceeds the physical memory size of a single machine, you need to build distributed clusters to expand the storage capacity.

Memcached does not support distribution per se, so distributed storage of Memcached can only be implemented on the client side using distributed algorithms such as consistent hashing.

Redis prefers to build distributed storage on the server side rather than on the client side. Distributed storage is already supported in the latest version of Redis.

Cache trible comparison

Cache advanced

Caching has been widely used in projects due to its high concurrency and high performance. This is especially true in highly concurrent, distributed, and microservice business scenarios and architectures.

High concurrency, distributed, and microservices all depend on high performance servers. When it comes to high-performance servers, you’re talking about caching.

High performance is mainly reflected in high availability, short service processing time and correct data.

Timely data processing is a “space for time” problem, using distributed memory or flash memory and other devices that can be accessed quickly to replace the database deployed on the general server, the file stored on the mechanical hard disk, this is the essence of caching to improve server performance.

High Concurrency: One of the most important things to consider when designing an Internet distributed system architecture. Usually, it refers to the idea of designing a system that can handle many requests in parallel.

Distributed: Efficiency is improved by shortening the execution time of a single task. For example, if a task consists of 10 subtasks and each subtask takes one hour to execute, it takes 10 hours to execute the modification task on one server. A distributed scheme is adopted to provide 10 servers, each server is only responsible for dealing with one sub-task, regardless of the dependencies between sub-tasks, and it only takes one hour to complete the task.

Microservices: The first key point emphasized by the architecture is that the business system needs to be thoroughly componentized and servitized. The original single business system will be split into several small applications that can be independently developed, designed, operated and maintained. These small applications interact and integrate with each other through services.

Cache consistency issues

How cache consistency occurs: write the database first, then flush the cache:

If the first step of writing to the database succeeds and the second step fails to flush the cache, a serious cache inconsistency problem will occur.Copy the code

How to avoid cache inconsistencies: Flush the cache first, then write the database:

If the first step succeeds in flushing the Cache and the second step fails to write to the database, only one Cache miss is raised.Copy the code

Distributed cache consistency

We use ZooKeeper to coordinate each cache instance node. Zookeeper is a distributed coordination service, including a primitive set, which can notify all clients of watch nodes and ensure that the sequence of events is consistent with the sequence of messages received by clients. This scenario can be easily implemented using the ZooKeeper cluster.

The consistent Hash algorithm uses a data structure called a consistent Hash ring to map keys to the cache server.

Cache avalanche

Cause 1.a. Because the Cache layer carries a large number of requests, it effectively protects the Storage layer (which is generally considered to be weak in pressure resistance). Therefore, the volume of Storage is actually low, so it is very cool. B. However, if the Cache level crashes for some reason (downtime, Cache service hangs, or does not respond), it means that all requests will reach the Storage level

Possible cause 2. The cache is set to expire at the same time. As a result, all cache requests are forwarded to DB.

The avalanche problem is referred to as the stampeding herd, referring to the tendency for traffic to crash in the cache.

The solution

  1. Locking/queuing ensures that the cache writes to a single thread

The avalanche effect of failure on the underlying system is terrible.

Most system designers consider locking or queuing to ensure that cached singleline (process) writes are kept from falling to the underlying storage system when a large number of concurrent requests fail.

Locking queuing is just to relieve the pressure on the database, and does not improve system throughput.

Suppose that in high concurrency, the key is locked during the cache rebuild, which is the last 1000 requests and 999 are blocked. It will also cause users to wait for timeout, which is a palliative method!

Distributed environment concurrency problem, may also solve the problem of distributed lock; Threads are blocked and the user experience is poor! Therefore, rarely used in true high-concurrency scenarios!

  1. Avoid simultaneous cache invalidation

Spread out the cache expiration time, for example, we can add a random value at the end of the original expiration time.

  1. Cache the drop

When traffic surges, service problems (such as slow or unresponsive response times) occur, or non-core services affect the performance of the core process, you still need to ensure that the service is still available, even at the expense of the service.

The system can automatically degrade according to some key data, or manually degrade by configuring switches.

The ultimate goal of a downgrade is to ensure that the core service is available, even if it is lossy. And some services can’t be downgraded (add to cart, checkout).

The system should be combed before demoting to see if the system can lose the pawn to protect the boss; To sort out what must be protected and what can be downgraded.

For example, you can refer to log level Settings:

(1) General: For example, some services can be automatically degraded due to timeout due to network jitter or online services;

(2) Warning: If the success rate of some services fluctuates within a period of time (such as between 95 and 100%), it can be automatically degraded or manually degraded, and an alarm can be sent;

(3) error: for example, the availability rate is lower than 90%, or the database connection pool was hit, or the number of visits suddenly jumped to the maximum threshold that the system can withstand, at this time can be automatically degraded or manually degraded according to the situation;

(4) Serious error: for example, due to special reasons, the data is wrong, and emergency manual downgrade is needed at this time.

Cache breakdown/cache penetration

Cache penetration refers to the query of a certain data does not exist, because the cache is passively written when the data is not hit, and for fault tolerance, if the data cannot be found from the storage layer, the data will not be written to the cache, so that the non-existent data will be queried to the storage layer every time, losing the significance of cache. During heavy traffic, DB may fail. If someone attacks our application frequently with non-existent keys, this is a vulnerability.

Cache penetration – Solution 1

As a simple and crude method, if a query returns null data (whether the data does not exist, or the system fails), we still cache the null result.

But its expiration time will be short, no more than five minutes.

Cache penetration – Solution 2

The most common is to use a Bloom filter, hash all possible data into a large enough bitmap, a non-existent data will be intercepted by the bitmap, thus avoiding the query pressure on the underlying storage system.

For example, if the mall has 1 million user data, swipe all user ids into a Map.

When the request comes in, check whether the Map contains the user ID. If it does not, the user id will be returned directly. If it does, the user ID will be checked in the cache first.

This will not only reduce the pressure on the database, but also greatly reduce the pressure on the cache system.

remarks

The ancients cloud: the paper come zhongjue shallow, and must know this to practice.

The experience and wisdom of others can only be verified by yourself before they can be used by us.

Other people’s knowledge is just some branches that you need to weave into a ladder to help you rise.

Refer to the link

  • Baidu Encyclopedia – cache
  • Spring Mind Map to make Spring easy to understand
  • importnew : Spring Cache
  • Illustrate the evolution of distributed architecture
  • EHCACHE
  • Ehcache official document
  • Ehcache Basic example
  • Ehcache detailed interpretation
  • Ehcache memcache Redis three cache tenors
  • Comparison of web caching technology Ehcache memcache Redis
  • Cache breakdown, failure, and hot key issues
  • Case study of service overload in Cache applications
  • Bloom Filter Bloom Filter
  • Caching is a common problem in high concurrency scenarios
  • Cache avalanche problem
  • Let’s talk about caching
  • Cache penetration problem
  • Design of microservitization cache
  • Ensure cache and database consistency
  • Distributed cache breakdown
  • CDN cache summary
  • Caching mechanism for integer types in AVA
  • Mysql query cache
  • Use Spring Session and Redis to solve the problem of distributed Session sharing across domains
  • Learn spring-Session +Redis to achieve Session sharing
  • Learn more about MySQL benchmarks and sysbench tools
  • Distributed cache breakdown
  • Resolution of distributed database and cache double-write consistency schemes