With more than a decade of Big Data experience now behind us, we’ll talk with a few veterans from the front lines about what skills mattered — and what didn't — on the first data engineering teams at Silicon Valley’s data-intensive start-ups. This will be a must-watch panel for those looking to build out a data-engineering function or get into the field themselves.
We’ll explore the build vs. buy debate, looking at which classes of software teams hand-rolled, borrowed (and extended) from open-source projects, or bought - and discuss how that mix is changing. We’ll learn about the hardest parts of data pipelines and the data stack to build. And we’ll highlight the necessary skills that make data engineers different than data scientists.
Ben Hamner is Kaggle’s co-founder and CTO. At Kaggle, he currently’s focused on creating tools that empower data scientists to frictionlessly collaborate on analytics and promote their results. He has worked with machine learning across many domains, including natural language processing, computer vision, web classification, and neuroscience.
Prior to Kaggle, Ben applied machine learning to improve brain-computer interfaces as a Whitaker Fellow at the École Polytechnique Fédérale de Lausanne in Lausanne, Switzerland. He graduated with a BSE in Biomedical Engineering, Electrical Engineering, and Math from Duke University.
Mike Driscoll founded Metamarkets in 2010 after spending more than a decade developing data analytics solutions for online retail, life sciences, digital media, insurance, and banking. Prior to Metamarkets, Mike successfully founded and sold two companies: Dataspora, a life science analytics company, and CustomInk, an early pioneer in customized apparel. He began his career as a software engineer for the Human Genome Project. Mike holds an A.B. in Government from Harvard and a Ph.D. in Bioinformatics from Boston University.
Vijay is current SVP, SW Engineering at Salesforce focusing on platform and infrastructure for Big Data, telemetry, visualization, IoT and ML/AI. In his last role, Vijay Gill was the GM and led teams for Microsoft Azure Global foundational infrastructure for Microsoft online services for consumers and businesses worldwide. This included design, engineering, operations, software development, data analytics, automation and capacity/demand planning as well as P&L for the service.
Sam Shah is cofounder and CTO of SkipFlag, which uses machine intelligence to build the knowledge graph for work. Previously, he was Director of Engineering at LinkedIn where he was responsible for consumer data products and infrastructure for the site including People You May Know, Who's Viewed My Profile?, Skills, and more. Sam holds a Ph.D. in Computer Science from the University of Michigan.