GFlowOut: Dropout with Generative Flow Networks
Dianbo Liu - Mila -- Quebec AI Institute
Nov. 4, 2022, 2:30 p.m. - Nov. 4, 2022, 3:30 p.m.
McConnell 11
Hosted by: Yue Li
Abstract: Bayesian Inference offers principled tools to tackle many critical problems with modern neural networks such as poor calibration and generalization, and data inefficiency. However, scaling Bayesian inference to large architectures is challenging and requires restrictive approximations. Monte Carlo Dropout has been widely used as a relatively cheap way to approximate Inference and to estimate uncertainty with deep neural networks. In this work, we propose GFlowOut GFlowOut which leverages the recently proposed probabilistic framework of Generative Flow Networks (GFlowNets) to learn the posterior distribution over dropout masks. We empirically demonstrate that GFlowOut results in predictive distributions that generalize better to out-of-distribution data, and provide uncertainty estimates which lead to better performance in downstream tasks.
Bio: Dianbo Liu is a postdoctoral machine learning researcher in Prof. Yoshua Bengio (Turing Award 2018) group, Mila-Quebec AI institute. He is also leading the humanitarian AI team of 19 researchers at Mila. Dianbo’s research spans both fundamental machine learning and its applications in biomedical informatics. Prior to joining the Bengio team, Dianbo worked and studied at University of Dundee, Harvard University and Massachusetts Institute of Technology. Dianbo co-found and served as the first CTO of Secure AI Labs, MA, USA, which is a MIT spin-off that focuses on federated learning for social good. In his personal life, Dianbo is a stand-up comedian in training.
