In 2018, Julien proposed a radical vision of databases as modular, recombinable systems—a concept I called the "Deconstructed Database." Today, that vision has crystallized into a transformative reality. This keynote explores how the big data ecosystem has evolved from a chaotic melting pot to a sophisticated, composable landscape driven by open-source standards. Attendees will journey through the emergence of critical technologies like Parquet, Arrow, Iceberg, and OpenLineage that are redefining data infrastructure. Learn how cloud technologies and decoupled compute-storage architectures are dismantling traditional data silos, creating a flexible ecosystem of specialized tools that promise unprecedented interoperability and freedom from vendor lock-in. Dive deep into the core components, architectural contracts, and innovative strategies that are reshaping the future of data engineering, offering a comprehensive blueprint for building adaptable, powerful data systems.
Julien Le Dem is the OpenLineage project lead at the LFAI&Data. He co-created the Parquet file format and is involved in several open source projects including OpenLineage, Marquez, Arrow, Iceberg and a few others. He was the Chief Architect of Astronomer and Co-Founded Datakin. Previously he held technical leadership positions at Wework, Dremio on the founding team, Twitter, where he also obtained a two-character Twitter handle (@J_); and Yahoo!, where he received his Hadoop initiation. His French accent makes his talks particularly attractive.