A, the body

As we all know, there are a lot of seckilling activities on tmall, JINGdong, Suning and other e-commerce websites. For example, when buying a mobile phone with the original price of 1999 and the price of only 999 in a second at a certain moment, there will be a peak of user requests, and there will be hundreds of thousands of concurrent amounts to grab this mobile phone. In high concurrency situations, the database server or file server application server will be under great strain, and in severe cases, may be down.

Another problem is that the amount of seconds kill things are, for example, a mobile phone is only the quantity of 10 seconds to kill, so, in the case of high concurrency, tens of thousands of article data update database (for example, 10 sets the amount of being robbed a would be recorded in the data set some minus 1), the order of this time is very mess, it is easy to appear the quantity of 10 sets, There were more than 10 of these serious problems. So, how do we solve the problems we are talking about? The techniques I’m going to share can be used to address these issues: distributed locks and task queues.

\

Second, implementation ideas

1.Redis implements distributed lock thinking

The idea is very simple, the main redis function is setnx(), this should be the most important function to achieve distributed lock. Insert Lock:order into redis by setnx(); insert Lock:order into Redis by setnx Return true if yes, false if no. Of course, it’s a little more complicated than that in my code, which I’ll explain when I analyze it.

2.Redis implements the task queue

The implementation here will use the above Redis distributed locking mechanism, mainly using the ordered collection data structure in Redis. For example, the add() function of Zset is used to join the team, and the getScore() function of zset is used to join the team. You can also pop up several tasks at the top.

That’s a simple way to implement distributed locking and task queuing. If you’re a little confused, look at the code implementation below.

\

Iii. Code analysis

First, analyze the code implementation of Redis distributed lock

1) To avoid lock failure due to special reasons, the lock will be given a lifetime (by setting the parameters of the lock method or using the default value) after the lock is successfully locked. The lifetime of the lock will be automatically released after the lifetime is exceeded. The default lifetime of the lock is relatively short (seconds). The expire method can be used to extend the lifetime of a lock to an appropriate time, such as within a loop.

2) System-level lock When a process crashes for any reason, the operating system will retrieve the lock itself, so resources will not be lost, but distributed lock is not used. If the process is set for a long time, when the unlock is not invoked due to a process crash or other exceptions, The lock becomes garbage in the remaining time, and other processes or processes cannot enter the locked area after being restarted.

\

The first is timeout, which is the waiting time for acquiring the lock in a cycle. During this time, it will keep trying to acquire the lock until it times out. If it is 0, it will directly return after the lock fails to be acquired without waiting any more. Another important parameter is timeout, which is the waiting time for the lock acquisition cycle. During this time, it will keep trying to obtain the lock until the timeout. If it is 0, it will directly return after the lock acquisition failure and no longer wait. Another important parameter is timeout, which is the waiting time for the lock acquisition cycle. During this time, it will keep trying to obtain the lock until the timeout. If it is 0, it will directly return after the lock acquisition failure and no longer wait. Expire is another important parameter. This parameter specifies the maximum lifetime of the current lock, in seconds. This parameter must be greater than 0. The most important function of this parameter is explained in (1) above.

\

We get the current time, and then we get the timeout (which is a timestamp) when the lock failed, and then we get the maximum time the lock can live. The redis key is in this format: Insert data into redis, using the setnx() function, which means insert data if the key does not exist, store the maximum lifetime as a value, and if insert succeeds, set the expiration time of the key. Place the key in the $lockedName array and return true.

\

If the key exists, the insert operation will not be performed. There is a serious step here, which is to obtain the remaining time of the current key. If this time is less than 0, the key does not exist, because setNx will create it automatically. Expire will not be called if an instance of a process crashes after setnx succeeds. If the wait time for lock failure is not set or the maximum wait time has been exceeded, the loop is quit, otherwise the request continues after $waitIntervalUs. This is the whole code analysis of locking.

/** * lock * @param [type] $name Lock identifier * @param INTEGER $timeout Specifies the timeout period for the lock to be acquired. * @param INTEGER $EXPIRE Specifies the maximum lifetime (in seconds) of the current lock. The value must be greater than 0. If the lifetime expires, the lock is not released. * @param INTEGER $waitIntervalUs Specifies the timeout interval (in microseconds) * @return [type] [description] */ public function lock($name, $timeout = 0, $expire = 15, $waitIntervalUs = 100000) { if ($name == null) return false; $now = time(); $timeoutAt = $now + $timeout; $expireAt = $now + $expire; $redisKey = "Lock:{$name}"; $result = $this->redisString->setnx($this-> rediskey, $expireAt); if ($result ! $this->redisString->expire($redisKey, $expireAt); $this->lockedNames[$name] = $expireAt; return true; $TTL = $this->redisString-> TTL ($redisKey); // If the TTL is less than 0, the key does not exist, because setnx creates it automatically. If ($TTL < 0) {$this->redisString->set($this-> redisKey); $expireAt); $this->lockedNames[$name] = $expireAt; return true; } / part request lock loop * * * * * * * * * * / / / if not set the lock wait time of failure or already exceeds the maximum waiting time, then exit the if ($timeout < = 0 | | $timeoutAt < microtime (true)) break; $waitIntervalUs ($waitIntervalUs); } return false; }Copy the code

Then look at the unlock code analysis: unlock is much simpler, pass in the parameter is the lock identifier, first determine whether there is a lock, if there is, from redis through the deleteKey() function to delete the lock identifier.

/** * unlock * @param [type] $name [description] * @return [type] [description] */ public function unlock($name) { If ($this->isLocking($name)) {if ($this->redisString->deleteKey("Lock:$name")) {$this->redisString->deleteKey("Lock:$name")) {$this->isLocking($name)  unset($this->lockedNames[$name]); return true; } } return false; }Copy the code

In the paste to delete all the lock method, in fact, are the same, more than a loop traversal.

Return [type] [description] */ public function unlockAll() {$allSuccess = true;  foreach ($this->lockedNames as $name => $expireAt) { if (false === $this->unlock($name)) { $allSuccess = false; } } return $allSuccess; }Copy the code

The above is to use Redis to achieve distributed lock the whole set of ideas and code implementation of the summary and share, here I attach is an implementation of the class code, code I basically annotated each line, convenient for everyone to quickly understand and can simulate the application. For a deeper look at the entire class code:

/** * class RedisLock {private $redisString; private $lockedNames = []; public function __construct($param = NULL) { $this->redisString = RedisFactory::get($param)->string; } /** ** lock * @param [type] $name Lock identifier * @param INTEGER $timeout specifies the timeout period in which the lock will be attempted until the timeout period. * @param INTEGER $EXPIRE Specifies the maximum lifetime (in seconds) of the current lock. The value must be greater than 0. If the lifetime expires, the lock is not released. * @param INTEGER $waitIntervalUs Specifies the timeout interval (in microseconds) * @return [type] [description] */ public function lock($name, $timeout = 0, $expire = 15, $waitIntervalUs = 100000) { if ($name == null) return false; $now = time(); $timeoutAt = $now + $timeout; $expireAt = $now + $expire; $redisKey = "Lock:{$name}"; $result = $this->redisString->setnx($this-> rediskey, $expireAt); if ($result ! $this->redisString->expire($redisKey, $expireAt); $this->lockedNames[$name] = $expireAt; return true; $TTL = $this->redisString-> TTL ($redisKey); // If the TTL is less than 0, the key does not exist, because setnx creates it automatically. If ($TTL < 0) {$this->redisString->set($this-> redisKey); $expireAt); $this->lockedNames[$name] = $expireAt; return true; } / part request lock loop * * * * * * * * * * / / / if not set the lock wait time of failure or already exceeds the maximum waiting time, then exit the if ($timeout < = 0 | | $timeoutAt < microtime (true)) break; $waitIntervalUs ($waitIntervalUs); } return false; } @param [type] $name [description] * @return [type] [description] */ public function unlock($name) { If ($this->isLocking($name)) {if ($this->redisString->deleteKey("Lock:$name")) {$this->redisString->deleteKey("Lock:$name")) {$this->isLocking($name)  unset($this->lockedNames[$name]); return true; } } return false; } @return [type] [description] */ public function unlockAll() {$allSuccess = true; foreach ($this->lockedNames as $name => $expireAt) { if (false === $this->unlock($name)) { $allSuccess = false; } } return $allSuccess; } /** * adds the specified lifetime to the current set, Must be greater than 0 * @param [type] $name [description] * @return [type] [description] */ public function expire($name, $expire = Max ($expire, 1); $expire = Max ($expire, 1); $expire = Max ($expire, 1); If ($this->redisString->expire("Lock:$name", $expire)) {return true; } } return false; } @param [type] $name [description] * @return Boolean [description] */ public function $locking ($name) {if ($this->lockedNames[$name]) {$locking ($name) {if ($this->lockedNames[$name]) (string)$this->lockedNames[$name] = (string)$this->redisString->get("Lock:$name"); } return false; }} Redis implements distributed lockingCopy the code

Two) Using Redis to achieve the task queue code analysis

1) Task queue, which is used to queue the operations that can be processed asynchronously in the business logic, and then queue out after processing in other threads

2) Distributed locks and other logic are used in queues to ensure the consistency of queue entry and queue exit

3) This queue is different from ordinary queues. The ID used to join the queue is used to distinguish repeated entries. There is only one record in the queue

First look at the team code analysis: First of all, of course, is to test the legality of parameters, and then use the above the contents of the locking mechanism, is to lock, team here I choose the current timestamp as a score, then there is the team, using the zset data structure of the add () method, after the completion of the team, to unlock the task, which finished the operation of a team.

Join a Task * @param [type] $name Queue name * @param [type] $ID Task ID (or its array) * @param INTEGER $timeout Queue timeout duration (s) * @param integer $afterInterval [description] * @return [type] [description] */ public function enqueue($name, $id, $timeout = 10 and $afterInterval = 0) {/ / legitimacy to detect the if (empty ($name) | | the empty ($id) | | $timeout < = 0) return false. // lock if (! $this->_redis->lock->lock("Queue:{$name}", $timeout)) { Logger::get('queue')->error("enqueue faild becouse of lock failure: name = $name, id = $id"); return false; $score = microtime(true) + $afterInterval; Foreach ((array)$id as $item) {if (false === = $this->_redis->zset->getScore("Queue: name", $item)) { $this->_redis->zset->add("Queue:$name", $score, $item); }} / / unlock $this - > _redis - > lock - > unlock (" Queue: $name "); return true; }Copy the code

If $score matches the queued Task, the queued Task will be queued again. If $score matches the queued Task, the queued Task will be queued again. First and the parameters of the legitimacy of the detection, and then use the function of the lock, and then out of the queue in time;

First use getScore() to get the score of this id from Redis, and then compare the score passed in with the score stored in Redis. If the two are equal, the queue operation will be performed, that is, delete the task ID by using the delete() method in Zset, and finally the task is unlocked. This is the code analysis of the exit team.

$id = $score; $score = $score; $score = $score; $score = $score; * * @param [type] $name Queue name * @param [type] $id task id * @param [type] $score task score * @param INTEGER $timeout Specifies the timeout period (s) * @return [type] Specifies whether the Task succeeded. False indicates that the redis operation failed. It could also be $score and the queue value does not match (this means that the Task by other threads since access to the local team) * / public function to dequeue ($name, $id, $score, $timeout = 10) {/ / legitimacy to detect the if (empty ($name) | | the empty ($id) | | the empty ($score)) return false. // lock if (! $this->_redis->lock->lock("Queue:$name", $timeout)) { Logger:get('queue')->error("dequeue faild becouse of lock lailure:name=$name, id = $id"); return false; $this-> _queue ->zset->getScore("Queue: name", $id); $result = false; If ($serverScore == $score) {if ($id == $result) (float)$this->_redis->zset->delete("Queue:$name", $id); if ($result == false) { Logger::get('queue')->error("dequeue faild because of redis delete failure: name =$name, id = $id"); }} / / unlock $this - > _redis - > lock - > unlock (" Queue: $name "); return $result; }Copy the code

Studied data structure’s friends should know, queue operations and pop up on the top of a value method and so on, processing operations team out of the team here, and I also implements the queue at the top of a number of Task and gives the method of team, would like to know friends can look at the code, if see not too clear message, here I am no longer to analyze it.

* @param [type] $name Queue name * @param INTEGER $count Number of tasks * @param INTEGER $timeout Timeout period * @return [type] returned array [0 = > [' id '= >' score '= >], 1 = > [' id' = > 'score' = >], 2 = > [' id '= >, 'score'=> ]] */ public function pop($name, $count = 1, $timeout = 10) {/ / legitimacy to detect the if (empty ($name) | | $count < = 0) return []; // lock if (! $this->_redis->lock->lock("Queue:$name")) { Log::get('queue')->error("pop faild because of pop failure: name = $name, count = $count"); return false; $result = []; $array = $this->_redis->zset->getByScore("Queue:$name", false, microtime(true), true, false, [0, $count]); Foreach ($array as $id =>$score) {$result[] = ['id'=>$id, 'score'=>$score]; $this->_redis->zset->delete("Queue:$name", $id); } / / unlock $this - > _redis - > lock - > unlock (" Queue: $name "); return $count == 1 ? (empty($result) ? false : $result[0]) : $result; }Copy the code

The above is the summary and sharing of the whole set of ideas and code implementation of the task queue with Redis. Here I attach the code of an implementation class, and I basically annotated each line in the code, so that we can quickly understand and simulate the application. For a deeper look at the entire class code:

/** */ class RedisQueue {private $_redis; public function __construct($param = null) { $this->_redis = RedisFactory::get($param); } /** * join a Task * @param [type] $name Queue name * @param [type] $ID Task ID (or its array) * @param INTEGER $timeout Queue timeout time (s) * @param integer $afterInterval [description] * @return [type] [description] */ public function enqueue($name, $id, $timeout = 10 and $afterInterval = 0) {/ / legitimacy to detect the if (empty ($name) | | the empty ($id) | | $timeout < = 0) return false. // lock if (! $this->_redis->lock->lock("Queue:{$name}", $timeout)) { Logger::get('queue')->error("enqueue faild becouse of lock failure: name = $name, id = $id"); return false; $score = microtime(true) + $afterInterval; Foreach ((array)$id as $item) {if (false === = $this->_redis->zset->getScore("Queue: name", $item)) { $this->_redis->zset->add("Queue:$name", $score, $item); }} / / unlock $this - > _redis - > lock - > unlock (" Queue: $name "); return true; $id = $score = $score = $score = $score = $score = $score = $score = $score; * * @param [type] $name Queue name * @param [type] $id task id * @param [type] $score task score * @param INTEGER $timeout Specifies the timeout period (s) * @return [type] Specifies whether the Task succeeded. False indicates that the redis operation failed. It could also be $score and the queue value does not match (this means that the Task by other threads since access to the local team) * / public function to dequeue ($name, $id, $score, $timeout = 10) {/ / legitimacy to detect the if (empty ($name) | | the empty ($id) | | the empty ($score)) return false. // lock if (! $this->_redis->lock->lock("Queue:$name", $timeout)) { Logger:get('queue')->error("dequeue faild becouse of lock lailure:name=$name, id = $id"); return false; $this-> _queue ->zset->getScore("Queue: name", $id); $result = false; If ($serverScore == $score) {if ($id == $result) (float)$this->_redis->zset->delete("Queue:$name", $id); if ($result == false) { Logger::get('queue')->error("dequeue faild because of redis delete failure: name =$name, id = $id"); }} / / unlock $this - > _redis - > lock - > unlock (" Queue: $name "); return $result; } @param [type] $name Queue name * @param INTEGER $count Number of tasks * @param INTEGER $timeout timeout period * @ return [type] returned array [0 = > [' id '= >' score '= >], 1 = > [' id' = > 'score' = >], 2 = > [' id '= >, 'score'=> ]] */ public function pop($name, $count = 1, $timeout = 10) {/ / legitimacy to detect the if (empty ($name) | | $count < = 0) return []; // lock if (! $this->_redis->lock->lock("Queue:$name")) { Logger::get('queue')->error("pop faild because of pop failure: name = $name, count = $count"); return false; $result = []; $array = $this->_redis->zset->getByScore("Queue:$name", false, microtime(true), true, false, [0, $count]); Foreach ($array as $id =>$score) {$result[] = ['id'=>$id, 'score'=>$score]; $this->_redis->zset->delete("Queue:$name", $id); } / / unlock $this - > _redis - > lock - > unlock (" Queue: $name "); return $count == 1 ? (empty($result) ? false : $result[0]) : $result; } @param [type] $name Queue name * @param INTEGER $count Number * @return [type] Return array [0=>['id'=>, 'score'=> ], 1=>['id'=> , 'score'=> ], 2=>['id'=> , 'score'=> ]] */ public function top($name, $count = 1) {/ / legitimacy to detect the if (empty ($name) | | $count < 1) return []; $result = []; $array = $this->_redis->zset->getByScore("Queue:$name", false, microtime(true), true, false, [0, $count]); / / will be foreach Task stored in an array ($array as $id = > $score) {$result [] = [' id '= > $id,' score '= > $score]; } return $count == 1? (empty($result) ? false : $result[0]) : $result; }} Redis implements the task queueCopy the code

To this, the two main functions of the basic explanation completed, for the task queue, you can write a shell script, let the server regularly run some programs, to achieve the queue out of the queue and other operations, here I will not combine it with the actual application to achieve, we understand the realization of the two functions can;

The code is written in PHP, and if you understand the implementation, you can use Java or.net or other languages to implement both functions. These two functions can be used in many scenarios, especially the second kill. Another one is snatching train tickets during the Spring Festival Travel rush. These two are the most vivid examples. Of course, there are many other applications, and I won’t list them all here.

The above content hopes to help you, more free PHP factory PDF, PHP advanced architecture video materials, PHP wonderful good article can be wechat search concerns: PHP open source community

2021 Jinsanyin four big factory interview real questions collection, must see!

Four years of PHP technical articles collation collection – PHP framework

A collection of four years’ worth of PHP technical articles – Microservices Architecture

Distributed Architecture is a four-year collection of PHP technical articles

Four years of PHP technical essays – High Concurrency scenarios

Four years of elite PHP technical article collation collection – database