Flink, which has gained popularity in recent years, is one of the most recognized big data computing engines; TIDB, as an open source NewSQL database, is also highly praised by the industry for its excellent horizontal expansion ability and high availability. So when Flink meets TIDB, what sparks spark?

Apache Flink Community Meetup Beijing Station is here again

On July 10, | Beijing | line

Flink x TIDB

This Meetup will be jointly organized by Flink Community and TIDB Community. As always, we have invited the technology giants of various industries. Five technical experts from Alibaba, TIDB, 360, Zhihu and netease will focus on Flink and TIDB. Share the best practices of how to implement accurate and stable real-time computing services, how to build real-time warehouse data link, how to use the characteristics of both to complete the closed-loop delivery of end-to-end real-time computing, as well as the inspiration and practice of Flink SQL and Flink-CDC architecture.

■ Highlights

A lot of practical dry products, Flink + TIDB combination can wipe out what kind of sparks, and let’s see the technical experts focus on real-time data business support, real-time warehouse, end-to-end real-time computing and other scenarios, with theory and practice to analyze the application of Flink + TIDB in each scene.

The activity forms are diversified. Offline and online activities are opened simultaneously. People in the same city can participate in the offline Meetup face-to-face communication, and people in other places can also watch the live broadcast online.

Rich surrounding waiting for you to take, sign up for the opportunity to get a lot of Flink community customized exquisite surrounding!

Special thanks to 360 for providing the venue

■ Event agenda

Sign up now

Apache Flink x TiDB Meetup · Beijing Station http://hdxu.cn/kF2ua

I. Introduction of guests and topics

Topic 1: JFlink on TIDB — Convenient and Reliable Real-Time Data Service Support

Lin Jia | netease Interactive Entertainment Technology Center real-time development engineer, Apache Flink Contributor

【 Introduction of Guests 】

Lin Jia is the person in charge of real-time business of netease interactive entertainment and billing data center, and the main program of real-time development framework JFLINK-SDK and real-time business platform JFLINK.

【 Introduction 】

Flink has become one of the hottest open source frameworks in real-time computing in recent years. TIDB is widely used in data center business due to its convenient and reliable HTAP fusion distribution characteristics. This sharing will discuss how to achieve accurate and stable real-time computing services under actual business scenarios based on the design philosophy of JFLINK-SDK real-time framework for data centers and the way to use TIDB, starting from the computing cases of data centers.

Topic 2: Flink + TIDB, Experience the Beauty of Real-time Database

Tianyi Wang | TIDB Community Department Architect

【 Introduction of Guests 】

Wang Tianyi, Architect of TIDB Community Department. I used to work in Fidelity Investment and Softbank Investment, and I have rich experience in designing high-availability database schemes, and I have in-depth studies on the high-availability architecture and database ecology of TIDB, Oracle, PostgreSQL, MySQL and other databases.

【 Introduction 】

The evolution of real-time data storehouse – Lambada architecture

  • Kappa architecture
  • At present, the basic technology stack architecture of the real-time data warehouse, TIDB + FLINK, establishes the data link of the real-time data warehouse
  • Infrastructure and extensibility of TIDB
  • TIDB + Flink Real-Time Database Architecture TIDB + Flink Best Practices

Topic 3: “End-to-end real-time computing for TIDB X Flink”

Sun Xiaoguang Basic R&D Architect of | Zhihu; TIKV Maintainer, Chairman of TIDB Community TOC

【 Introduction of Guests 】

Sun Xiaoguang, architect of Zhihu basic research and development team, has been engaged in distributed system related research and development for a long time, focusing on cloud native technology.

【 Introduction 】

Flink is widely used for real-time computation in high real-time scenarios, while TIDB is widely used in OLTP transaction scenarios in large data scale. TIDB’s excellent horizontal scaling capability makes it an ideal downstream database storage for Flink real-time computing tasks, which can better meet the storage requirements of real-time computing results with high throughput and low delay. And Flink’s strong expansion ability can also provide sufficient computing capacity for the TIDB cluster with high throughput and writing continuously, and provide sufficient guarantee for the real-time performance of large traffic data calculation. In the past, when native TIDB Flink streaming computing power was lacking, users had to take a roundabout approach to stream or batch TIDB data using tools designed for MySQL. This not only increases the extra burden of learning, but also makes it difficult to make full use of the unique architectural features of TIDB to improve the overall efficiency of the system. In the process of transition from the overall online database system of Zhihu to TIDB, we urgently feel the importance of TIDB X Flink native batch stream integrated solution. This sharing will introduce some of Zhihu’s work in TIDB x Flink batch stream integration, and take the practical business as an example to introduce how to make full use of the characteristics of the two to complete the closed-loop delivery of end-to-end real-time computing.

Topic 4: Flink SQL Practices in 360

Zhang Chaoming | Apache Hudi Contributor, 360 big data platform R&D engineer

【 Introduction of Guests 】

Zhang Chaoming is a R&D engineer of 360 big data platform. I love technology and am very interested in big data-related technology, especially the principle and development of real-time computing. Currently working in 360 Systems Department, I am involved in the implementation and promotion of real-time computing related technologies in the department, and responsible for the productization and optimization of Flink SQL.

【 Introduction 】

With the continuous development of the company’s search, finance, advertising, games and other businesses, it has become an urgent need to promote faster iteration of business value through real-time computing. As the main real-time computing engine promoted in the company, Flink is increasingly used in the company. In order to accelerate development efficiency and reduce learning costs for developers, we actively encourage users to use Flink SQL for job development.

This sharing will focus on the product implementation process of Flink SQL in 360, and share some of our insights, practices and improvements on Flink SQL.

Topic 5: A detailed explanation of Flink-CDC

Xu Bangjiang | Apache Flink Contributor, Alibaba Senior Development Engineer

【 Introduction of Lecturer 】

Xu Bangjiang (XueDu) is a senior development engineer at Alibaba, focusing on the development of Flink SQL engine.

【 Introduction 】

How to connect the data in the database to the data warehouse/data lake is a key link that needs to be considered in the construction of the data warehouse. This sharing compares the advantages and disadvantages of the traditional data synchronization scheme and the Flink-CDC data synchronization scheme, analyzes the advantages of the Flink-CDC architecture, shares the Flink-CDC 2.0 design scheme, and explains the lockless design and full-phase concurrent design in detail.

II. Activity registration

■ Activity details

Time: 13:30-17:30, July 10

Venue: Room A, Building 360, Building 2, Yard 6, Jiuxianqiao Road, Chaoyang District, Beijing

Registration link: https://1712399719478.huodong…

Watch live: https://developer.aliyun.com/…

Offline space is limited, first come, first served! For more technical questions about Flink, please visit the Flink Learning Chinese website: https://flink-learning.org.cn/