For high concurrency architecture, there is no doubt that cache is the most important link. For a large number of high concurrency, we can use a three-tier cache architecture, nginx+ Redis + Ehcache
nginx
For middleware Nginx is often used to distribute traffic, and Nginx itself also has its own cache (limited capacity), which can be used to cache hot data, so that users’ requests directly go through the cache and return, reducing the flow of traffic to the server
Template engine
Often we can use a template engine such as Freemaker/Velocity to handle a large number of requests
Small system may directly apply colours to a drawing gives all the pages on the server and in cache, then the same page request can return directly, don’t have to query data sources or data logic processing For pages it systems, when the template is changed, the above method will need to render all page template, there is no doubt that it is not advisable. Therefore, with nginx+ LuA (OpenResty), the template is stored separately in the Nginx cache, and the data used for rendering is also stored in the Nginx cache, but a cache expiration time is required to ensure the real-time performance of the template as much as possible
2. Two-layer Nginx to improve cache hit ratio
For multiple NginX deployments, not incorporating some routing policy for data can result in a poor cache hit ratio per Nginx. Therefore, two-tier NGINx can be deployed
Distribution layer nginx is responsible for the logic and policy of traffic distribution according to some rules it defines, such as hash according to productId. Then modded the number of back-end Nginx requests to a fixed route to a backend Nginx server used to cache some hot data into its own cache (can the distribution layer only configure 1)
redis
If nginx does not cache the corresponding data, it will enter redis cache. Redis can cache the full amount of data, which can improve the concurrency and high availability ability through horizontal expansion
Persistence mechanism: Persist data in redis memory to disk, and then upload disk files to S3(AWS) or ODPS (Ali Cloud) and other cloud storage services on a regular basis.
If both RDB and AOF persistence mechanisms are used, when Redis restarts, AOF will be used to rebuild data. Because the data in AOF is more complete, it is recommended to enable both persistence mechanisms and use AOF to ensure that data is not lost as the first choice for data recovery. RDB is used for varying degrees of cold backup to quickly recover data when AOF files are lost or corrupted and unavailable.
If you want to recover data from the RDB and the AOF switch is on, you can’t recover the data because the AOF switch is on first. [redis config set appendonly Yes] [redis config set appendOnly Yes] [redis config set appendOnly Yes] [redis config set appendonly Yes] [redis config set appendonly Yes] [redis config set appendonly Yes] [redis config set appendonly Yes] Open the AOF configuration and start the data again
1.RDB
Data in Redis is persisted periodically, each moment a snapshot of the full amount of data is persisted. It has little impact on redis performance and can quickly recover anomalies based on RDB
2.AOF
Write to a log file in appends-only mode. When Redis restarts, the entire data set can be rebuilt by playing back the write instructions in the AOF log. (In fact, the log data is written to the Linux OS cache first, and redis calls the operating system fsync every second to write the data from the OS cache to disk). It has certain performance impact on Redis and can ensure data integrity as far as possible. Redis uses the rewrite mechanism to ensure that AOF files are not too large, based on current in-memory data and can be refactored appropriately.
2. Redis cluster
1.replication
The master node is responsible for writing, and synchronizes data to other Salve nodes (asynchronous execution), and the slave node is responsible for reading, which is mainly used for horizontal expansion of read and write separation architecture. In this architecture, the data of the master node must be persisted. Otherwise, when the master is down and restarted, the memory data will be emptied, and the empty data will be copied to the slave, causing all data to disappear
2. Sentinal sentry
Sentinel is an important component in the redis cluster architecture. It is responsible for monitoring whether the Master and slave processes of Redis work properly. When a Redis instance fails, sentinel can send alarm messages to the administrator and automatically transfer to the slave node when the master node fails. If failover occurs, the client client is notified of the new master address. Sentinal needs at least three instances to be robust on its own, and to be better able to vote for quorum to reach majority for failover. The biggest characteristic of the first two architecture modes is that the data of each node is the same, so massive data cannot be accessed. So sentinel clustering is a way to use with small data volumes
3.redis cluster
The Redis Cluster supports multiple master nodes. Each master node can mount multiple slave nodes. If mastre fails, the corresponding slave node will be automatically switched to master. It should be noted that under the Redis Cluster architecture, the slave node is mainly used for high availability and fault master/slave switchover. If it is necessary for the slave to provide read capability, the configuration can also be modified (at the same time, the Jedis source code needs to be modified to support read/write separation operation in this case). In the Redis Cluster architecture, the master can be expanded at will. Directly expanding the master horizontally can improve read and write throughput. Slave nodes can be automatically migrated (master nodes should have slave nodes on average as much as possible), so that the redundant slaves can guarantee higher availability of the system.
ehcache
Tomcat JVM heap memory cache, mainly against redis large-scale disaster. If There is a massive redis outage that causes a lot of nginx traffic directly to the data production service, then the final Tomcat heap memory cache can also handle some requests, avoiding all requests going directly to DB
I have specially sorted out the above technologies. There are many technologies that can not be explained clearly by a few words, so I simply recorded some videos with my friends. The answers to many questions are simple, but the thinking and logic behind them are not simple. If you want to learn Java engineering, high performance and distributed, simple. Micro services, Spring, MyBatis, Netty source analysis of friends can add my Java advanced group: 694549689, group ali Daniel live explain technology, and Java large Internet technology video free to share with you.
Cache data update policy
For the cache data with high timeliness requirements, when the change occurs, the database and Redis cache double-write scheme is directly adopted to make the cache with the highest timeliness.
For time-sensitive data, when changes occur, MQ asynchronous notification is adopted. MQ messages are monitored by the data production service, and the data of the service is asynchronously pulled to update the Tomcat JVM cache and redis cache. After the nginx local cache expires, new data can be pulled from Redis and updated to nginx local.
Cache aside pattern
1. Read data from the database and put it into the cache. At the same time, a response is returned
2. Update, delete the cache first, and then update the database only delete the cache is updated, because for some complex logical cache data, update data changes every time a cache can cause extra burden, just delete the cache, the next is used to make the data to perform the read operation to cache, The lazy loading strategy is used here. For example, if a table field is changed 20 times in a minute, or 100 times, then the cache is updated 20 times, 100 times; However, the cache is read once in a minute, so there is a lot of cold data in the cache every time it is updated. For the cache, the 28 golden rule, 20% of the data, accounts for 80% of the visits
Database and Redis cache double write inconsistency problem
1. The most elementary cache inconsistencies and solutions
Problem: If you modify the database first and then delete the cache, when the cache deletion fails, the database will have the latest data and the cache will still have the old data, causing data inconsistency.
Solution: You can delete the cache first and then modify the database. If the cache is deleted successfully but the database modification fails, the database is old and the cache is empty without inconsistency
2. Analysis of complex data inconsistency problems
Question: for data to change, first remove the cache, then go to change the database, the data in the database has not been modified successfully, concurrent read requests come to cache was found empty, and then to the database query to the old data in the cache, then before the database data to modify success, can cause data inconsistency
Solution: Asynchronously serialize database and cache update and read operations. When data is updated, the update operations are routed to a queue within the JVM based on the unique identity of the data. Each queue corresponds to a worker thread, which executes the operations sequentially from the queue. When performing operation in the queue to update data, delete the cache, then go to update the database, this time is not yet complete update when a read request, read empty cache you can cache update first, after the request is sent to the routing of the queue at this backlog in the queue, and then synchronization waiting for cache update is complete, It does not make sense to string multiple requests for the same data cache update in a queue, so it can be filtered. Wait until the previous update data operation completes the database operation before the next cache update operation is performed. At this point, the latest data is read from the database and written to the cache. If the request is still within the waiting time range, Polling finds that it can fetch a value from the cache and return it directly (there may be multiple requests for the cached data being processed at this point); If the request waits for an event for more than a certain amount of time, the request reads the old value directly from the database
There are a few things to note about this approach:
1. Read the request long block: as the read requests to a very mild asynchronous, so the timeout problems need to be pay attention to, more than timeout will query the DB directly, processing is bad to can cause pressure on the DB, so need to test the system peak QPS to adjust the machine number as well as the corresponding machine queue eventually decided to reasonable requests waiting timeout
2. Request routing for multi-instance deployment: It is possible that multiple instances of the service will be deployed, so the corresponding requests must be routed to the same service instance through the Nginx server
3. Tutor request skew for routing hot data: Since the cache will be emptied only when commodity data is updated, and then concurrent read and write operations will occur, if the update frequency is not too high, the impact of this problem is not particularly significant, but it is possible that the load on some machines will be higher
Distributed cache reconstruction concurrency conflict resolution
For the cache production service, it may be deployed on multiple machines. When redis and EhCache corresponding cache data are expired, the request from Nginx may arrive at the same time as the request from Kafka listening. As a result, both nginx and Kafka end up pulling data and storing it in Redis. The distributed lock similar to Redis or ZooKeeper can be used to solve the problem, so that the passive cache reconstruction of the request and the active cache reconstruction operation can be monitored to avoid concurrent conflicts. When storing in the cache, the old data can be discarded by comparing the time field and the latest data can be stored in the cache
Cache cold start and cache warm-up solutions
When the system is started for the first time and a large number of requests flood in, the cache at this time is empty, which may lead to DB crash and make the system unavailable. This problem also occurs when all redis cache data is abnormally lost. Can in advance into the data to the redis, therefore, to avoid the cold start problem, of course, also can’t be full amount data, can according to the specific access, similar to the real time statistics of higher thermal data access frequency, data in here also is more, need more service parallel distributed to read and write in the redis (so be zk based distributed lock)
Using Nginx + Lua to report access traffic to Kafka, Storm consumes data from Kafka and counts the number of times each item is accessed in real time, based on the memory data structure LRU (Apache Commons Collections LRUMap) storage scheme. LRUMap is used for storage because of its high performance in memory and no external dependence. When each Storm task is started, it writes its ID into the same zK node based on the ZK distributed lock. Each Storm task is responsible for completing the statistics of its own hot data and traversing the map at intervals. Then maintain a top 1000 data list, update the list, and finally start a background thread to synchronize the top 1000 hot data list to zK every once in a while (say one minute) and store it in a Znode corresponding to this Storm task
When deploying multiple instances of the service, it will get the node data of the storm Task ID list maintained above, and then try to obtain the ZK distributed lock of the Znode corresponding to taskid one by one according to taskid. If the distributed lock can be obtained, If it is not preheated, the hot data list corresponding to the taskid will be retrieved from the DB and written into the cache. If the taskid distributed lock fails to be obtained, the hot data list corresponding to the taskid will be retrieved from the DB and written into the cache. The distributed lock of the next TASkid can be obtained through a quick error toss cycle. At this time, multiple service instances coordinate and parallel for cache preheating based on zK distributed lock
Cache hot spots causing system unavailability solution
A sudden influx of a large number of requests for the same data may cause the corresponding application layer nGINx to be overwhelmed after the hash strategy. If the request continues, other NGINX will be affected, and eventually all NGINX will become abnormal and the entire system will become unavailable.
To solve this problem, the traffic distribution strategy based on nginx+ Lua + Storm’s hotspot cache is automatically degraded. You can set the data that is accessed n times more than the average of the next 95% as the hotspot, and send HTTP requests directly to the nginx in Storm to store the data in the local cache. Storm will also send the full cache of hotspot data to all application nginx servers and store it directly in the local cache. For the traffic distribution Nginx, access to the corresponding data, if it is found to be a hotspot identifier, the traffic distribution policy is immediately degraded, the access to the same data from the hash to one application layer nginx to be distributed to all application layer Nginx. Storm needs to save the last identified hotspot List and compare it with the current calculated hotspot List. If it is no longer a hotspot, it will send the corresponding HTTP request to nGINx to cancel the hotspot identification of the corresponding data
Cache Avalanche solution
Redis cluster thoroughly collapse, cache service request waiting for a lot of redis, takes up resources, then the cache service a large number of requests into the source to query the DB services, make the DB collapse pressure, at this time also a lot of waiting to take up to a request for source service resources, cache services a large number of resources all cost during a visit to redis and source service to nothing, Finally, they can’t provide services, and eventually the whole site will crash.
The solution in advance is to build a set of highly available architecture of Redis Cluster, with a master-slave architecture, one master and many slaves. Once the master node goes down, the slave nodes automatically follow up, and it is best to deploy the cluster with two machine rooms.
The solution in question is to deploy a layer of EhCache that can withstand some of the pressure if redis is fully implemented; Perform resource isolation on access to Redis Cluster to avoid all resources waiting and access to Redis Cluster failure to deploy the corresponding circuit breaker policy and redis Cluster degradation policy. Flow limiting and resource isolation of access to source services
After the solution: Redis data backup can be directly restored, restart Redis can be; If the redis data fails completely or is too old, you can quickly warm up the cache and let Redis restart. Then, because the half-open policy of resource isolation finds that Redis is reachable, all requests are automatically resumed
Cache penetration solution
If there is no corresponding data in the multi-level cache and the DB does not query the data, a large number of requests will directly reach the DB, leading to the problem of high concurrency. The problem of cache penetration can be solved by returning data with a null identity for data that the DB does not have, which can be stored in caches at all levels. Because there is asynchronous listening for data modification, when data is updated, new data will be updated to the cache sink.
Nginx cache failures cause redis stress
Nginx can set a random cache validity period for data cached locally to avoid a large number of requests going directly to Redis when both caches fail at the same time
This process is worth studying and thinking deeply.