Sun Xiaoguang, head of Zhihu Technology Platform team, is responsible for the construction of Zhihu’s online and offline infrastructure platform to provide a unified infrastructure for business development. Engaged in private cloud related product development for many years, focus on cloud native technology, TiKV project Committer.
Followers of TiDB may have noticed that following Follower Read’s PR merge on TiKV, tiDB-related PR has recently been merged into the trunk. If all goes well with the later stability tests, the functionality should ship with TiDB 3.1. The Follower Read function itself is small in terms of the amount of code, but the implications of this feature are huge, especially for Internet-type services.
Recently, PingCAP CTO Dongxu Huang has written a very thorough introduction and analysis of Follower Read from the perspective of principle. Today, I would like to introduce the implementation process of the Follower Read function from a non-technical perspective, and talk about my personal experience and feelings during the TiDB development process from the perspective of Contributor. Finally, from the perspective of zhihu technology platform team, let’s talk about our desire and determination to actively participate in open source projects and jointly build the community in the future.
The story behind Follower Read
Follower Read has been on the minds of PingCAP engineers for a long time, but for various reasons this feature has not been a high enough priority to make it into the development plan. In the middle of this year, we began to try to introduce TiDB into the wider business of Zhihu. In the gray scale process, we encountered the problem that TiDB performance was not ideal under certain workloads. At present, it seems that TiDB read and write operations are all handed over to the Leader, which is the bottleneck point that we encounter throughput problems under specific loads.
After clarifying our thinking, we communicated with PingCAP engineers for several times and reached an agreement. We decided to use Follower Read to solve the throughput problem of extreme hot data access in our business scenarios. Driven by the actual demand, PingCAP prioritized Follower Read’s related features, quickly determined the technical solution of related functions and started the development of TiKV terminal. As the demander of Follower Read function, Zhihu is responsible for the implementation of this demand on TiDB.
It took PingCAP engineers about two weeks to complete the development and test of the whole function of Follower Read on TiKV layer and merge it into the Master branch. Then we started the development of the corresponding function on TiDB side. Finally, with the help of PingCAP friends, the corresponding functions were merged into the Master branch. As the result of the first joint development between Zhihu and PingCAP, the implementation of this function is of great significance to both sides.
From the Contributor to the Committer
Before contributing to TiDB as a member of Zhihu technology platform team, I have made some small contributions to TiKV & TiDB as a user in the past year. Next, I would like to talk about the growth process and experience from Contributor to Committer from a personal perspective.
The first TIME I submitted PR and became Contributor for a PingCAP project happened about a year ago, when I tried to add some batch operation ports to TiKV while choosing the right storage backend for an internally developed business system. After PR was proposed, Tang Liu, chief architect of PingCAP, established contact with me immediately. In subsequent communication, he helped me improve PR quickly and finally merged into the main branch of TiKV.
Although I have submitted patches to many open source projects on a piecemeal basis in the past, this experience was completely different. The enthusiastic help and encouragement of PingCAP and community partners made me personally feel the unique charm of an active open source community. Subsequently, out of personal interest, I made some simple contributions in TIDB-related projects one after another, and was finally recognized as one of the committers of TiKV project.
Embrace open source
Looking back, the growth and harvest during this period are huge. I not only learned how to communicate more efficiently with many excellent contributors in the open source community, but also had a deeper understanding of the value concept of open source and the great significance of open source in the field of basic software.
Recently, with the change of the team in the company, I began to pay more attention to the evolution of the online and offline infrastructure in Zhihu. Zhihu has always encouraged the embrace of open source and built its technical architecture based on a large number of open source components. Thanks to the power of open source, its internal platform has always been closely following the trend of industrial technology development. In the past, we used to draw nutrients from the open source community from the perspective of users. With the growth of Zhihu’s technical architecture and internal engineering capabilities, we hope to participate in open source projects in a more active manner and give back to the community in the future. Follower Read is the first time for Zhihu to participate in TiDB community construction as a Contributor. In the future, we will participate in more technical community construction and contribute our strength to the development of open source community.
pingcap.com/blog-cn/zhi…
