Event
Alumnus Talk: "The Hidden Gift of Quadratic Landscapes: No Spurious Local Minima"
Friday, May 24, 2019
10:30 a.m.
2460 A.V. Williams Building
Kara Stamets
301 405 4471
stametsk@umd.edu
Alumnus Talk: "The Hidden Gift of Quadratic Landscapes: No Spurious Local Minima"
Speaker: Abbas Kazemipour, Departments of Electrical Engineering and Neurobiology, Stanford University
Hosted by Professor Min Wu
Friday, May 24, 10:30 am
2460 AV Williams
Abstract: Despite their practical success, a theoretical understanding of the loss landscape of neural networks has proven challenging due to the high-dimensional, non-convex, and highly nonlinear structure of such problems. In this talk, we provide optimal theoretical guarantees, namely the absence of spurious local minima, on three fundamental non-convex optimization problems, thus guaranteeing the convergence of stochastic gradient descent to a global minimum. The first is a two-layer neural network with quadratic activations where the number of hidden neurons is equal or greater than the input dimension and the output dimension is of arbitrary size. The second is the same network structure where the number of neurons is equal to the rank of the data, and the third is the related problem of low-rank matrix completion. Our theoretical results fill the existing gap in theory for two-layer neural networks and generalize results on matrix completion from positive definite matrix completion to general matrix completion under minimal assumptions on the training data or observations. Finally, we demonstrate empirically convergence to a global minimum on all three problems.
Speaker's Bio: Abbas Kazemipour received the PhD and M.Sc degrees in Electrical Engineering from University of Maryland, College Park in 2017, and the B.Sc. in Electrical Engineering from University of Tehran in 2012. His PhD thesis titled “Compressed Sensing Beyond the IID and Static Domains: Theory and Applications” won the Distinguished Dissertation Award from the Department of Electrical & Computer Engineering at UMD. His work as a research specialist at Janelia Research Campus from 2016-2018 broke a fundamental physical limit on the speed of fluorescence microscopy. He is currently a post-doctoral fellow at the departments of Electrical Engineering and Neurobiology at Stanford University where he does research on the optimization theory and machine learning with applications to biological neural networks.
