preface

After someone in a wechat group offered 50,000 yuan for Golang version of matchmaking engine, I wanted to develop one of my own. After all, in my experience, it is not difficult to develop a high-performance matchmaking engine.

So, in my spare time, I slowly developed a Golang version of the high-performance matching engine. It took about a month to go back and forth. Again think oneself haven’t updated article for a long time, my personal IP has been rusty, also should send big action to grind a grind. Therefore, I decided to share how I designed and implemented this matchmaking engine worth more than 50,000 yuan in serial format.

Originally, want to send into the nugget booklet, collect some payment, after all, this is a software with great commercial value, but asked the personnel of the nugget, they do not accept this kind of topic at present. Finally decided to release free of charge, can also multiple channels, perhaps also can bring me more attention.

All right, let’s get to the business of matching engines.

Introduction to Matching engine

Match engine is the core component of all dealmaking system, whether it’s stock trading system, including the spot trading, futures and options trading, etc., or digital currency trading system – including COINS COINS, contracts, such as leveraged deals, as well as a variety of precious metals trading systems, commodity trading system, etc., Although the trading objects of different trading systems are different, as long as they all adopt matching trading mode, they are inseparable from matching engine.

The matching engine can have universality, a set of matching engine with universality can theoretically be applied to any matching trading system, without any code adjustment. That is to say, the same matching engine can be used in stock trading system, can also be used in digital currency trading system, can be used for spot trading, can also be used for contract trading.

So, a set of universal matching engine should have what function? Before determining the answer to this question, let’s take a quick look at what a complete transaction process looks like. The following steps are generally included:

  1. The system opens the trading function of a trading object.
  2. The user submits the trading declaration of the subject matter of the transaction, that is, the order.
  3. The system verifies whether the order is valid, including whether the trading object is in the state of trading, whether the price and quantity of the order meet the requirements, etc.
  4. Determine Maker rate and Taker ** rate for the order.
  5. Check the status of the user’s asset account, including whether the account status is restricted, whether there is enough money to place an order, etc.
  6. The detailed order data is persisted to the database and the corresponding amount of funds in the user account is frozen.
  7. The order is matched, that is, the order can be matched with the order in ** transaction OrderBook **, the result of matching may be: all, part or no match. When all or part of the transaction is completed, there may be one or more matching orders in the transaction entrust book, that is, one or more transaction records will be generated. When there is no match or partial transaction, part of the order data, including the remaining number of uncompleted transactions, will be temporarily stored in the transaction entrust book, waiting to be matched with the subsequent order.
  8. The transaction records generated by matching are persisted to the database, and the market data is generated according to the historical transaction records, such as K line data, today’s rise and fall, etc.
  9. Update order data for all transaction orders in the database, as well as the asset account balance of the order user.
  10. Send the updated order data and market data to the front desk.

The whole transaction process involves many services, including user service, account service, order service, matchmaking service, market data service and so on. Only step 7 is handled by the matching engine. From the single responsibility principle, the matching engine should only do one thing, and that is to be responsible for matching orders. Order persistence before matching, freezing funds, etc., and generating K-line data after matching, etc., should not be the responsibility of matching engine.

Matching bidding

There are generally two kinds of matching bidding, one is set bidding, the other is continuous bidding. The stock trades the system to be able to use different bidding method in different trading time period commonly, for example when open or close, use set bidding, produce opening price or closing price thereby, other time uses continuous bidding. In most digital currency trading systems, there is no collective bidding, only continuous bidding, and the opening price is usually set before trading begins.

Call auction

So-called collection bid, it is to point to the sale that receives inside a period of time order of power of sale of a lump sum the bidding way that makes a match. Take the stock trading system of Shenzhen and Shanghai as an example, the time for collective bidding is between 9:15 and 9:25 of each trading day. During this period, the order received by the system will not be executed immediately. Instead, all orders will be sorted according to the principle of price priority and time priority, and on this basis, a benchmark price will be found to satisfy the following three conditions:

  1. The price at which the maximum volume can be achieved;
  2. The price at which all orders above the price and all orders below the price can be completed;
  3. The price at which at least one of the buyers or sellers will complete the transaction.

At the end of 9:25 a.m., the benchmark price is determined to be the transaction price, and all orders above that price and all orders below that price will be traded at that price. Failed to close the order, automatically transferred to continuous bidding.

But what if there are two or more prices that meet these three conditions? In this regard, the shenzhen Stock Exchange and The Shanghai Stock Exchange have different treatment plans. The Shenzhen Stock Exchange will take the price closest to the previous closing price as the transaction price, while the Shanghai Stock Exchange will take the price with the smallest untraded volume as the transaction price. If there is still more than one price with the smallest untraded volume, the middle price will be the transaction price.

The main purpose of a rally bid is to determine the opening or closing price.

Continuous bidding

So-called continuous bidding, also is the bidding way that we are familiar with, it is to point to the bidding way that successive match is made to buy and sell a proxy. The user’s hanging order, want to satisfy clinching a deal only the condition, can clinching a deal immediately. And the collection bid, wait until the last minute will clinched a deal.

In continuous bidding, the principle of price priority and time priority should still be met:

  1. Price priority: pay the higher price can be the priority to clinch a deal, sell order is the lower price can be the priority to clinch a deal.
  2. Time priority: the purchase and sale direction and the same price of the order, the order will be declared first than after the declaration of order priority transaction.

In addition, the bid price must be greater than or equal to the ask price to make a deal. When the bid price equals the ask price, the transaction price is the bid price or ask price. When the bid price is greater than the ask price, the latest bid price is determined by reference to the previous bid price. Suppose the bid price is B, the ask price is S, the previous bid price is P, and the latest bid price is N, then:

  • If P >= B, then N = B
  • N = S if P <= S
  • If B > P > S, N = P

A general matching engine should support both bidding methods, but for the same trading subject, two bidding methods cannot be carried out at the same time, so it is necessary to consider how to switch between the two bidding methods in the design. The specific implementation ideas will be discussed in the following chapters.

Quality requirements

In addition to the functional requirements mentioned above, our matchmaking engine should also meet some quality requirements, especially high requirements for availability, scalability and performance. In addition, in order to achieve generality, reusability needs to be met.

First of all, reusability, what we expect is that this matching engine can be used for both stock trading systems and digital currency trading systems, for both coins and contracts. Therefore, the matching engine should avoid introducing business logic that is strongly related to the specific system to enhance its reusability.

Looking at performance, the performance of a matchmaking engine is measured by how well it can process TPS per pair, that is, how many orders of the same pair can be processed per second. Previously, based on database matching technology, TPS was usually only 10 strokes per second. Nowadays, the TPS can easily reach 1000 strokes per second, and 10,000 strokes per second or even higher using a proprietary high-performance server.

Moving on to scalability, each of our matchmaking engines can handle either multiple transactions at once or just a single transaction. When the amount of transaction objects and concurrent transactions increases, servers can be added and deployed as matching engine clusters to process different transaction objects, so as to achieve load balancing.

Finally, talk about availability, high availability is mainly reflected in two points, one is the failure rate to be low, the second is the failure maintenance time to be short. To reduce the failure rate, the matching engine needs to have a high robustness, for a variety of abnormal conditions that may lead to engine failure to consider and design a good solution. In addition, multi-server hot backup can be used to improve availability, and to ensure data consistency between the servers, it is necessary to introduce memory state machine replication scheme, which is much more complicated to implement.

However, we do not have to reach high quality requirements all at once, because the higher the requirements, the more complex the architecture and implementation. We can start with a simple version and then iterate.

summary

Our goal is to achieve a universal matching engine that supports collective bidding and continuous bidding, as well as some quality requirements to improve the system’s reusability, performance, scalability, availability and so on. Subsequent chapters will delve deeper into the design and implementation of these requirements. In addition, we will design and implement multiple versions of matchmaking engine in the way of continuous upgrading and iteration.

Leave two questions for consideration:

  1. At the end of the bid, if there is no benchmark price that meets the three conditions, how will the opening price be determined?
  2. For a single transaction pair, can performance be improved by horizontally adding servers?

Scan the qr code below to follow the subscription number.