In the digital era, the technology iteration and updating is more rapid than ever before, and the database technology also bursts out a huge development tension in the tide of technology, and the kernel technology as the soul of database technology has also realized a transition development.
Huawei cloud database has always attached importance to the development of kernel technology, has rich experience in database kernel development, and continues to build a series of highly competitive kernel features. Huawei’s new-generation financial distributed database GaussDB(for openGauss) has launched several new kernel features, such as the Ustore storage engine and PAxos-based DCF high availability component, to create a stable and high-performance storage engine and high availability kernel capability for enterprise users. With innovative technology as the starting point, further promote the digital transformation of enterprises.
Ustore storage engine innovation, the performance is still stable under frequent data updates
Ustore Storage engine, also known as in-place Update storage engine. In-place Update is a new storage mode for GaussDB(for openGauss) kernel. The GaussDB(for openGauss) kernel currently uses the Append Update mode as the row engine. Append updates work well for business additions, deletions, and HOT(Heap only tuple) updates, but garbage collection is not efficient for non-HOT Update scenarios across data pages.
And Ustore engine will be the latest version of the “valid data” separation and historical versions of junk data storage, the latest version of the “valid data” is stored in the data page, and open up a UNDO space alone, used for unified management history version of “garbage data”, so the data space is not due to the frequent updates and expansion, garbage collection efficiency is higher. Ustore engine adopts numA-aware UNDO subsystem design, which makes UNDO subsystem can be effectively extended on multi-core platform. At the same time, the multi-version index technology is used to solve the index cleaning problem and effectively improve the efficiency of storage space recycling.
Figure 1: Ustore data storage diagram: Data in the original page, undo through pointer management separately
In conclusion, compared with the original garbage data cleaning mechanism, Ustore data storage can bring higher performance, higher efficiency, more full space utilization, more stable operation of the overall system, and adapt to more business scenarios and workloads.
In terms of application practice, THE Ustore storage engine has been POC tested in some customers’ sites, showing stable and reliable database service ability and achieving the expected effect.
High availability DCF component innovation based on Paxos protocol ensures strong data consistency and high availability
In addition to the Ustore storage engine, GaussDB(for openGauss) also introduces the DCF high availability component based on the Paxos protocol, which enhances GaussDB(for openGauss) in high availability while ensuring data consistency. The components include:
(1) Get rid of third-party arbitration components through self-arbitration and majority selection of master, greatly shorten the RTO time, and can prevent the split brain under any failure of double master;
(2) Support the multi-cluster deployment mode of synchronous and asynchronous node deployment;
(3) Improve the log replication efficiency between the active and standby nodes to improve the maximum throughput capacity of the system.
Figure 2: Paxos principle diagram
With the DCF high availability component of GaussDB(for openGauss), users can not only avoid the risk of system cracking, but also improve availability.
In the future, GaussDB(for openGauss) will continue to build more intelligent, secure and efficient technical capabilities in key features such as Ustore storage engine and Paxos protocol optional master based on AI autonomous technology, and start from the most urgent problems and long-term needs of thousands of industries database applications. We will continue to make breakthroughs in core technologies to create more advanced and better database services for customers.