SQL is the lingua franca of data processing, and everybody working with data knows SQL. Apache Flink provides SQL support for querying and processing batch and streaming data. Flink's SQL support powers large-scale production systems at Alibaba, Huawei, and Uber. Based on Flink SQL, these companies have built systems for their internal users as well as publicly offered services for paying customers. In my talk I will show how to leverage the simplicity and power of SQL on Flink.
I’ll explain why unified batch and stream processing is important and what it means to run SQL queries on streams of data. Once we’ve covered the basics, I will spend the remainder of the talk demonstrating the capabilities of Flink SQL. We will explore different use cases that Flink SQL was designed for by running queries on Flink’s SQL shell. In particular, I will demonstrate the unified batch and streaming engine by running the same query on batch and streaming data and show how to build a real-time dashboard that is powered by a streaming SQL query, which continuously updates an external result table.
Timo Walther is a long-term member of the management committee and among the top committers in the Apache Flink project. He studied Computer Science at TU Berlin and participated in the Database Group. Timo worked as a software engineer at Data Artisans and lead of the SQL team at Ververica. He was a Co-Founder of Immerok which was acquired by Confluent in 2023. In Flink, he is working on various topics in the Table & SQL ecosystem to make stream processing accessible for everyone.