This is the 19th day of my participation in the Genwen Challenge

What is idempotency

• Idempotent definition:
  • One and more requests for a resource should have the same result for the resource itself
  • Any number of executions has the same impact on the resource itself as a single execution
• Key points of idempotent definition:
  • Idempotent is not just one or more requests that have no side effects on the resource
    • For example, query database operations, no add, delete or change, no matter how many operations have no impact on the database
  • Idempotent also means that the first request has side effects on the resource, but subsequent requests do not have side effects on the resource
  • Idempotent concerns whether subsequent requests have side effects on the resource, not the result
  • Network timeouts are not idempotent
• Idempotence is a promise, not an implementation, of a system service
• It promises that multiple external calls will have the same effect on the system as long as the calling interface succeeds
• A service declared idempotent assumes that external invocation failures are the norm and that retries are inevitable after failures

Idempotent usage scenarios

• In business development, repeated commits are common:
  • The request was reinitiated because the result could not be received due to a network problem
  • Front-end operation jitter caused by repeated commits
• In the case of transaction systems, the problem caused by the repeated submission of payment systems is particularly obvious:
  • If the user clicks on the APP repeatedly to submit an order, only one order should be generated in the background
  • Make a request to the payment system. If the request is reissued due to network problems or system bugs, the payment system should only make one deduction operation
• Idempotent services believe that there will be multiple invocations by external callers. In order to prevent multiple invocations from changing the data state of the system, the service needs to be idempotent

Idempotent and anti-weight

• The case of repeated commits is different from the original intention of service idempotent
  • A repeat commit is an artificial multiple operation that causes a service that does not meet idempotent requirements to change state multiple times after the first request has already been successful
  • The more idempotent use is when you make multiple requests for the first time because you don’t know the result or the request fails because of something worse than timeout
• The purpose of idempotent is to make multiple requests to confirm the success of the first request without multiple state changes due to multiple requests

The case where idempotency is guaranteed

• In SQL, there are three scenarios, and only the third one needs to be idempotent:
  • SELECT col1 FROM tab1 WHERE col2=2
  • UPDATE tab1 SET COL1 =1 WHERE col2=2 UPDATE tab1 SET col1=1 WHERE col2=2
  • UPDATE tab1 SET col1=col1+1 WHERE col2=2

Design idempotent services

• Idempotent makes the client logic simple, but the server logic complex
• Meeting idempotent services requires two pieces of logic:
  • The last execution status is queried first, if not, it is considered the first request
  • Ensure that the logic against repeated commits is preserved before the service changes the business logic of the state

Ensure idempotent strategy

• Idempotent needs to go throughUnique business order numberTo ensure that:
  • If the order number is the same, the same service is considered
  • Use a unique order number to ensure that the processing logic and execution effect of the same order number are consistent
• Example idempotent implementation – Pay:
  • First check whether the order has been paid
  • If the payment has already been made, return the payment succeeded
  • If no payment is made, the payment process is carried out and the status of the order is changed to paid

Anti-duplicate submission policy

• In an idempotent strategy, execution is performed in two steps, with the latter step depending on the query results of the previous step, thus atomicity cannot be guaranteed
• The inability to guarantee atomicity is problematic in cases of high concurrency:
  • The second request is made when the next order status of the first request has not been changed to “paid status”
  • To solve this problem, lock the query and change-state operations, and change the parallel operations to serial execution

Optimistic locking

• If you are just updating existing data, there is no need to lock the business
• Use optimistic locks when designing table structuresversionTo implement optimistic locking:
  • Ensure execution efficiency
  • Ensure the idempotent

  UPDATE tab1
  SET	col1=1,version=version+1
  WHERE	version=#version# 
Copy the code

Since THE ABA problem can lead to the failure of optimistic locks, the ABA problem will not occur as long as the version value increases

The heavy table

• useorderNoAs a unique index in the de-redo table, each request is based on the order numberorderNoInsert a row into the undo table:
  • First request to query order payment status:
    • The order has not been paid
    • Conduct payment operation
    • Update the status of the order to success or failure after execution, regardless of success or failure, and delete data from the repeat table
    • Subsequent orders return the operation failed until the first request completes (success or failure) because the unique index in the table failed to be inserted
• The function of anti – heavy table is to achieve the function of locking

A distributed lock

• You can use Redis distributed locks instead of anti-replay tables
• Example:
  • The order initiates a payment request
  • The payment system will check the Redis cache for the existence of the order Key
  • If not, add Key to Redis as the order number
  • Check whether the order payment has been paid
  • If not, the payment will be made, and the Key of the order will be deleted after the payment is completed
• Distributed locking is implemented through Redis, and the next request will not come in until this order request is completed
• Redis distributed locks are placed concurrently in the cache, which is more efficient than de-table
• Only one payment request can be completed at a time

Token token

• The token is divided into two phases:
  • Token Application stage:
    • Before entering the order submission page, the order system initiates a request for token application to the payment system according to the user information
    • The payment system saves the tokens in the Redis cache for use in the payment phase
  • Payment stage:
    • The order system obtains the requested token, initiates the payment request,
    • The payment system checks Redis for the presence of thistoken
      • If yes, it indicates that the payment request is initiated for the first time, and the token in the cache is deleted to start payment logic processing
      • If the request does not exist in the cache, it indicates an invalid request

Payment buffer

• Payment buffer:
  • Order payment requests are quickly received down, is a fast request to receive the buffer pipeline
  • Use asynchronous tasks to process data in the pipeline and filter out duplicate data to be paid for
• Advantages: Synchronous to asynchronous, high throughput
• Disadvantages: unable to return payment results in time, need to listen for asynchronous return of payment results later

Disadvantages of idempotent: - Idempotent is to simplify the client logic, but increase the logic and cost of the service provider - the use of idempotent needs to be analyzed according to the specific scenario - additional control idempotent business logic is added, and business functions are complicated - parallel functions are converted into serial, reducing the execution efficiencyCopy the code