Suvrit Sra (MIT)
Non-convex optimization through a geometric lens abstract
Abstract:
Nonconvex optimization is currently enjoying intense interest in machine learning. This interest is largely fueled by deep learning, and also by more complex tasks in related areas. Although an understanding of why neural networks work so well remains elusive, there has been impressive progress in algorithms, software, and systems for nonconvex optimization.But in today\'s talk, I want to take a step back from algorithmic advances (fast nonconvex SGD, escaping saddle-points, adaptive methods, etc.). Instead, I want to draw your attention to a new set of tools that expand the repertoire of tractable nonconvex models. In particular, I will present a rich subclass of nonconvex problems that can be solved to global optimality by leveraging geometry.The specific concept that I\'ll discuss is geodesic convexity, which generalizes the usual vector-space notion of convexity to nonlinear spaces (more generally, I will also cover geodesically non-convex problems). My aim is to outline how geometric thinking leads to improved models or insights for fundamental tasks such as large-scale PCA, metric learning, Gaussian mixture models, among others. I will outline both theoretical and practical aspects, including iteration complexity theory, and conclude with some open problems.
11:00 • ETH Zentrum, Building ML , Room F 34