Taming the Deep Learning Workflow

Deep learning has yielded a step function improvement at an array of important problems ranging from computer vision to natural language processing, and there is enormous excitement about its potential. However, building practical applications powered by deep learning remains an enormous challenge: the necessary expertise is scarce, the hardware requirements can be prohibitive, and current software tools are immature and limited in scope.

In this talk, we will first describe how deep learning workflows are supported by existing software tooling. We will then describe several promising opportunities to drastically improve these workflows via novel algorithmic and software solutions, including automated hyperparameter optimization, efficient utilization of distributed resources via performance models, and reproducible workflow management. This talk draws on academic work done at CMU, Berkeley, and UCLA, as well as our experiences at Determined AI, a startup that provides software to make deep learning engineers dramatically more productive.