This article is published by Tencent Cloud database
Note: This article is excerpted from the speech of Tencent cloud database CynosDB’s new launch on November 22, 2018. With the development of Internet information, we are not particularly unfamiliar with the word cloud, as one of the world’s preferred cloud service manufacturers, Tencent Cloud, relying on Tencent’s years of business accumulation and technology precipitation, we launched a high-performance, low-cost, safe and reliable overall data service solutions. In addition, with the increasing of customer data, relevant research and analysis have been done, and a new generation of self-research database of Tencent Cloud has been launched. Here I first sell a mystery, by our side of the technical guests to reveal our relevant database content.
—————————— I’m the splitter ————————
Good afternoon, everyone! I am the product manager of Tencent Cloud Database Product Center NewSQL. My name is Liu Feng. I think anyone here knows what NewSQL is? I’ll give you a narrow definition, NewSQL is just SQL plus NoSQL. There are three important phases in the history of databases. The first is the introduction of the SQL standard, the second is the popularity of NoSQL, and the third is the birth of NewSQL. If you missed the first two important historical periods, then this moment is the third important moment, the birth of Tencent Cloud research NewSQL database.
Let’s start by looking at what types of cloud databases are available. There are two types, relocation and cloud native. With the advent of the era of cloud computing, manufacturers need to provide customers with cloud database services. A relatively direct idea is that, as a cloud manufacturer, we can move the traditional database to the cloud and provide customers with services in the way of renting. This is how most of our cloud vendors provide services. When we did this, we found that the traditional database has a lot of problems in the cloud, and these problems are traditional database existing problems, in the cloud has not been solved. At the same time, we found that traditional databases could not adapt to the advantages of cloud scalability, reliability, and scale. At this time, we are forced to cloud manufacturers to consider how to do a fusion of traditional database and cloud database, cloud computing, this fusion point, we call cloud native.
Take a more appropriate example, we can understand that the traditional database moved to the cloud to provide cloud database services is a move mode, and the cloud native database is a “carry the bag in” way.
We mentioned earlier that traditional databases have a lot of problems. Now let’s take a look at some of the problems with traditional databases.
The first problem is that the backup file is slow. This is a classic backup file structure using traditional database in the cloud. To make a backup file, we need to copy the original data to the cloud storage and copy the cloud storage data to the user database entity. It takes 3 hours to use 1T data and 6 hours to use 2T data. If 3T data is backed up and refiled, I can go off work this evening.
The second problem is that the failure recovery is slow. The possibility of a separate database support is very low. If the database goes down, the business will be interrupted. In general, we have a slave database hanging next to the primary database, and the slave database will synchronize data from the primary database, but unfortunately, in most cases, if your primary database is under a lot of pressure, the slave database will not be able to get timely updates. An inappropriate example is that during the shopping rush of Double 11, you choose a lot of gifts from shopping websites and put them in the shopping cart, but the main database happens to be down. Even though there is a secondary database, you do not get the updated data from the main database in time, so you will find that those things in your shopping cart have disappeared. We think this kind of slow recovery is a serious problem in traditional databases.
The third and fourth problem is that vertical expansion is weak and horizontal expansion is weak. A traditional database is a box that holds data, and if you want to scale a traditional database you need a bigger box. An example more image, such as our own developed a game, in the beginning we thought the game play with not many users, we use a database is a CPU 100 g, 2 g memory storage space, luckily I sold this game big, then find the CPU memory use, need to scale, What does a traditional database do? Create a 4-core, 8GB, 1TB database and move the database here. This involves the problem of database migration, so 1T data for 3 hours, 2T data for 6 hours, 3T data I can not finish today, and then tomorrow. Believe that this service provided to customers, customers will not be satisfied with the same horizontal expansion is the same problem.
The fifth problem is high maintenance cost. Why are maintenance costs high? If using traditional database, both to artificially manual data backup retracement, and to participate in the horizontal and vertical expansion, then make fault recovery, artificial participation cost is very high, at the same time, combined with the original structure of efficiency is very low, is also contributing to the time cost and artificial cost is very high, so that our maintenance cost is very high.
The sixth problem is low resource utilization. Traditional databases are big boxes with cpus, memory, and hard disks inside. But do I need a lot of CPU, memory and hard disk? This is not always the case. Sometimes we only need a higher CPU and larger memory, and sometimes we need a larger hard disk. It is difficult to meet the different needs of users in the three dimensions that is the big problem of traditional database.
Number seven, low availability. An example of the last month, a buyout by Microsoft, and the price is a global social code programming and hosting sites, due to it with the traditional client-server database did high availability in the two data centers, the government replaced the two centers in the United States in the middle of the cable, interrupted, 43 seconds is the 43 seconds caused the entire site within 24 hours of downtime and paralysis. As you can imagine, with traditional database architecture, even such high-tech companies have encountered the problem of service unavailability, let alone the enterprises around us.
In order to solve the problem of traditional database, as a cloud service manufacturer, we provide a complete one-stop solution — Database product CynosDB developed by Tencent Cloud. CynosDB comes from the word Cynosure, which is the name of the north star in ancient Greek mythology, the brightest star in the little Diursa. CynosDB is a high-performance, high-availability, enterprise-level cloud database developed by Tencent Cloud, compatible with open source databases MySQL and PostgreSQL.
Let’s take a look at how we thought about designing such a database. Using the popular term “C-bit”, we considered c-related design features at the beginning of the design of CynosDB.
CynosDB core design concept:
First, it is Comprehensive and compatible with the new open source databases MySQL and PostgreSQL, while most manufacturers in the industry can only be compatible with one. We are compatible with two major open source databases, which can help smooth migration for enterprises.
Second, Creative divide-and-rule. The traditional database architecture is actually a big box, how to allocate the memory, CPU and disk reasonably, reduce the waste of resources? Our approach is divide-and-conquer, separating computing from storage. Don’t put it in one box, put it in multiple boxes. With this structure, we need to communicate between the computing layer and the storage layer. Communication is carried out through the network, which makes us need to reduce the traffic of network communication, so Tencent’s CynosDB implements a “log is database” architecture.
Third, Cloud Native should be born in the Cloud. Our database is built on the existing efficient and stable cloud services of Tencent Cloud, so we can quickly build a set of cloud database with high performance, high availability and high reliability.
Fourth, it is Cost Effective. We need a cloud database that can outperform traditional databases in terms of performance, but also reduce the cost pressure for customers, because the essence of cloud computing is to give customers a very affordable service, so we are talking about real pay-as-you-go and flexible expansion.
Fifth, Cohesive exists. Minimalist software optimization unleashing hardware dividends. We always care about the iPhone launch every year, in fact, every year apple talks about the fact that we have higher CPU this year, we have more memory, have you ever thought that one day, our iphones will run slower and slower, but Apple tells you that through software optimization, Let the original iPhone4, iPhone5, iPhone ese, do not need to upgrade to a faster CPU, larger memory can use all the App store? Maybe Apple will do it one day, but CynosDB has done it today.
Now take a look at CynosDB’s product architecture based on the core design philosophy. We split the box, we separate the computing layer, and the storage layer is implemented with distributed storage that comes with the cloud. The communication between the computing layer and the storage layer is carried out through the network. Our architecture is truly compatible with multiple database engines, because when we separate the computing layer, our computing layer is compatible with multiple database engines. After the separation of computing and storage, storage is an independent service, we can understand that computing is a layer of services, storage is a layer of services, we are connected to the cloud network security services, different dimensions to ensure security and reliability. We provide customers with software optimization combined with new hardware solutions, can be based on ordinary hardware to meet their high performance, high availability requirements. If there are more requirements, because we are an enterprise-class database, we can combine new hardware, such as our SSD based SPDK driver and RDMA network card to provide higher performance database services. We give the option to the customer.
Therefore, we say that the CynosDB product architecture is a database architecture based on cloud services and ordinary hardware, and we release the hardware dividend without passing on the hardware cost.
Then we will introduce a core component of CynosDB called CynosFS computable intelligent storage, which is a storage system created by Tencent. We are the first in China to implement the “log as database” architecture. What does that mean? What you see on the left is actually an architecture of a traditional database. When a write operation occurs in a traditional database, many IO writes are required between computation and storage. I had to do the same IO between storage and backup, which added a lot of load to the system. CynosDB is the right approach, we reduce many IO merge to log only, so we call it log is database, that is, in our system only exists in the data log, we call it minimalist IO. This is equivalent to transforming Beijing’s busy Second Ring Road into a comprehensive, three-dimensional rail transit system, which has greatly improved the efficiency of the system. In addition, CynosDB also realized the operation and maintenance automation of the storage, you can see that the log can be returned after writing, but at the bottom of the storage, we automatically save the data into three copies, is maintained through the distributed consistency protocol. In special cases, if an error occurs in one copy we automatically copy data from the other two copies to the broken copy to repair the data. At the same time, the blue part is our data page, if the system needs to access the data page, we will be obliged to convert logs into data, and this system has realized automatic expansion and shrinkage, so we call it the distributed storage system of operation and maintenance automation.
Let’s take a look at CynosDB’s ability to take advantage of this original architecture and unique product creation.
First, we are the first cloud database in the industry to be compatible with multiple database engines.
Second, the performance broke through the one million mark, to achieve the same level of rapid performance of commercial databases.
Third, the storage capacity has reached 128TB, realizing twice the performance and storage capacity of the leading foreign cloud database manufacturers.
Fourth, say goodbye to the minute and enter the second era. Various pain points of the database have been mentioned before, such as slow recovery and uncontrollable master/standby switchover. CynosDB, on the other hand, improves the recovery time of the entire compute node and faulty node from minute to second. After separating the computing layer from the storage layer, we can achieve fast recovery and make the failed nodes highly available.
Fifth, second level backup, just mentioned the time required by backup and file back 1T data, CynosDB has made a great improvement in this pain point, snapsh-based backup and file back, achieved a leap of second level, 180 times faster than the traditional backup and file back.
Speaking of these product capabilities, let’s take a look at CynosDB’s user value, which is what users care about most.
First, it reduces costs and provides a database product with high cost performance ratio. We did this with minimal optimizations based on common hardware and software. To achieve performance and availability comparable to commercial databases at about 1/15 of the cost.
1) In terms of explicit cost, cloud storage reduces the cost of hardware procurement, and our cost is an order of magnitude lower than that of foreign commercial databases. 2) After CynosDB separates the storage computation, the storage is truly pooled. As we all know, scale can reduce the boundary cost, and produce marginal effect, we realize the storage through distributed computing storage real serverless architecture, realize a storage layer of intelligence, greatly reduce the cost of storage. That is, you can allocate your costs based on how much you use, not like we used to need a box or a bigger box. 3), just now we mentioned that we have realized many automatic operation and maintenance functions to reduce manual intervention, which is equivalent to reducing the hidden cost of our users.
Combining the three dimensions of cost savings, our cost is only 1/15 of the traditional commercial database.
Second, flexibility. If you have the experience of buying a database in the current cloud, you will know that if you buy a database, the first is how much CPU you need, the second is how much memory you need, and the third is how much hard disk you need. CynosDB does not need these three steps, do not need to know how much hard disk you want, after the purchase of 10T, pay 10T, need to pay hundreds of tons of money, if you do not need to store can not spend money. Found true automatic scaling and pay-as-you-use.
Third, smooth the clouds. Fully compatible with MySQL5.7 and PostgreSQ10, two currently updated versions of open source databases, the original application with zero changes, enabling smooth migration and fast cloud access for enterprises.
Fourth, safe and reliable. I’m going to give you three dimensions of how we can be safe and reliable. This is a lifeline for our cloud database and cloud services. 1) In the longitudinal dimension, CynosDB storage implements an automatic multi-version maintenance. As I mentioned just now, when we write one copy of data in our storage, we can automatically maintain three copies of data, and in the longitudinal dimension, more copies of data are added. 2) We provide cross-regional high availability, you can add more copies to another data center area, even if the data in this data center is down, that data can still be provided, we provide high availability, further guarantee security and reliability; 3) It also supports the existing security group and VPC of Tencent Cloud for network isolation. The gateway service of Tencent Cloud can withstand hacker attacks and DDOS attacks.
We put many of the user values we’ve just listed into the context of how we can solve enterprise problems.
First, high-performance, highly available enterprise applications. We can smooth the cloud, ensure the continuous and efficient operation of the business, reduce the operating cost of the enterprise, and reduce manual intervention through automatic operation and maintenance. Here’s a little bit of a sneak peak. Our computable intelligent storage is going to be in the realm of artificial intelligence, and we’re going to minimize human intervention. So I say that CynosDB is a natural fit for traditional enterprises, supporting the key core business of traditional enterprises.
Second, the Internet and gaming business. 1) Deal with peak business. My colleague also introduced that no matter it is e-commerce or games, the Internet business often has a great change of peak business volume. CynosDB’s flexible computing extensions are naturally suited to peak business. In games, we do a lot of campaigns for online users. Traditionally, operations would buy equipment a month in advance and then write it off after the event, but CynosDB doesn’t have to do that. 2) Quick backup and file back. As I mentioned earlier, we have increased the speed of backloading by 180 times. You can imagine that when you have a problem with your version of the game, you can fix the problem faster than your competitors, and you can provide a faster service to your customers. 3) Massive scalable storage, CynosDB is a distributed shared storage, game companies can use it to achieve mixed service and service, at the same time there is no need to use multiple instances to support the game business. So CynosDB is a natural fit for Internet companies, games, and applets.
[Tencent cloud 100 yuan without the threshold of vouchers to rob] everyone new and old folks, Tencent cloud database year-end benefits come! In order to better provide services for the vast number of developers, data jun sincerely invite you to participate in this prize research, say what you want to say, use the most useful database! The questionnaire will take you about two minutes. After answering the question, you will have the chance to get the Tencent Cloud 100 YUAN no-threshold voucher, which can only be used by the cloud database and cloud server. If you win the prize, please fill in your mobile phone number on the page, which will be used to receive the voucher code. Limited number of first come, first served, the entry will be closed on December 15, click to participate!
This article has been authorized by the author to Tencent Cloud + community, more original text pleaseClick on the
Search concern public number “cloud plus community”, the first time to obtain technical dry goods, after concern reply 1024 send you a technical course gift package!