Scaling Deep Learning
Jeff Dean - Google
Dec. 10, 2014, 2 p.m. - Dec. 10, 2014, 3 p.m.
McGill University M1 amphitheater of the Strathcona - 3640 University Street
Three years ago we started a small effort to see if we could build training systems for large-scale neural networks and use these to make significant progress on various perceptual tasks. Since then, the project has been used by dozens of different groups at Google to train state-of-the-art models for speech recognition, image recognition, various visual detection tasks, language modeling, ads click prediction, and various other tasks. In this talk, I'll highlight some of the distributed systems and techniques that we use in order to train large models quickly. I'll then discuss ways in which we have applied this work to a variety of problems in Google's products, usually in close collaboration with other teams.
Jeff joined Google in 1999 and is currently a Google Senior Fellow in Google's Knowledge Group. He has co-designed/implemented five generations of Google's crawling, indexing, and query serving systems, and co-designed/implemented major pieces of Google's initial advertising and AdSense for Content systems. He is also a co-designer and co-implementor of much of Google's distributed computing infrastructure, including the MapReduce, BigTable and Spanner systems, protocol buffers, LevelDB, systems infrastructure for statistical machine translation, and a variety of internal and external libraries and developer tools. He is currently working on large-scale distributed systems for machine learning. Prior to joining Google he worked for Digital Equipment Corporation's Western Research Lab, and for the World Health Organization's Global Programme on AIDS. He is a Fellow of the ACM, a Fellow of the AAAS, a member of the U.S. National Academy of Engineering, and a recipient of the Mark Weiser Award and the ACM-Infosys Foundation Award in the Computing Sciences. He received a B.S. in computer science & economics, summa cum laude, from the University of Minnesota, and a M.S. and Ph.D. in computer science from the University of Washington.
