Operating Kafka as a service, at scale, is one challenge. Operating multi-tenant Kafka as a service, at scale, brings all kinds of new challenges. How do you determine the optimum packing density? How do you define and enforce resource limits so all tenants can coexist? How do you handle a large number of partitions? Learn how we worked our way through these problems, formed the opinions built into our service today, and designed a product that developers love. And see what some of our most advanced data-driven customers are doing with Kafka and Heroku.
Ali Hamidi is a Lead Engineer on the Heroku Data team at Salesforce where he works on the core control plane that manages 2 million data services. Previously he worked at 140 Proof where he architected and built an interest inference platform that analyzed billion node social graphs in real-time.