Data Council Blog

According to a recent article published in The New York Times, water damage from hurricane Harvey extended far beyond flood zones. Now that the rescue efforts are underway, it’s clear that much of the damage occurred outside of the typical boundaries drawn on official FEMA flood maps.

Alyssa Frazee tells us about the unicorn data skills she's honed on the job.

How open source allowed Airbnb to rebuild their expensive BI tool in less than one developer year

Granted Maxime Beauchemin isn't your average data engineer. As any Bay Area engineer worth their salt knows, anyone who worked on data at Facebook receives (deserves) a certain outsized respect from their peers.

How Everyone's Favorite PaaS Operates Kafka at Scale

Coinbase is on the front-lines of discovering advanced cryptocurrency and payment fraud techniques. Hear about how they use machine learning to help them fight the war.

Druid is a modern data store built for analytics use-cases. As the volume of data has exploded, and companies have sought deeper insights from their data, ad-hoc analytics have become difficult as more data is buried in distributed systems like Hadoop & Spark. The query model for these systems can result in long latencies making them sub-optimal for interactive analytics applications.

How many different file formats does your ETL system need to parse? For many data pipelines, several well-defined formats will suffice. Things break, and at times require manual intervention, but not so often that a couple engineers can't keep tabs on the system and keep things running relatively smoothly.

Shopping for geeks can be stressful, especially if you don’t know what to get someone. Take a data scientist or engineer for example: do you get him or her gadgets, clothing, or an experiential gift? Should it be practical or fun? How about familiar or unique?

Conference swag is great, but cash is better. This year, the sponsors of DataEngConf NYC '16, which ncluded over 25 data science and engineering experts from top data, finance and media companies, decided to award one luck open source software project $2,000, plus free tickets to the next DataEngConf in San Frnacisco in Spring, 2017, in lieu of conference swag.