Clark School Home UMD

ISR Events Calendar

Event Information

CCPS Seminar: Varun Jog, "Information Theoretic Perspective on Learning Algorithms"
Thursday, November 15, 2018
5:00 p.m.-6:30 p.m.
2168 AV Williams Building
For More Information:
Ajaykrishnan Nageswaran
301 405 3661

Communication, Control and Signal Processing Seminar

Information Theoretic Perspectives on Learning Algorithms

Varun Jog
Assistant Professor
Electrical and Computer Engineering Department
Fellow, Grainger Institute for Engineering
University of Wisconsin

In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. We overview some recent work [Xu and Raginsky (2017)] that bounds generalization error of empirical risk minimization based on the mutual information I(S;W) between the algorithm input S and the algorithm output W. We leverage these results to derive generalization error bounds for a broad class of iterative algorithms that are characterized by bounded, noisy updates with Markovian structure, such as stochastic gradient Langevin dynamics (SGLD). We describe certain shortcomings of mutual information-based bounds and propose alternate bounds that employ the Wasserstein metric from optimal transport theory. We compare the Wasserstein metric-based bounds with the mutual information-based bounds and show that for a class of data generating distributions, the former leads to stronger bounds on the generalization error.

Varun Jog received his B.Tech. degree in Electrical Engineering from IIT Bombay in 2010, and his Ph.D. in Electrical Engineering and Computer Sciences (EECS) from UC Berkeley in 2015. Since 2016, he is an Assistant Professor at the Electrical and Computer Engineering Department and a fellow at the Grainger Institute for Engineering at the University of Wisconsin - Madison. His research interests include information theory, machine learning, and network science. He is a recipient of the Eli Jury award from the EECS Department at UC Berkeley (2015) and the Jack Keil Wolf student paper award at ISIT 2015.


This Event is For: Graduate • Undergraduate • Faculty • Post-Docs

Browse Events By Calendar

Calendar Home

« Previous Month    Next Month »

May 2019
1 2 3 4 w
5 6 7 8 9 10 11 w
12 13 14 15 16 17 18 w
19 20 21 22 23 24 25 w
26 27 28 29 30 31 w

Search Events

ISR lecture and seminar series

Distinguished Lecturer Series
Intelligent Automation Inc. Colloquia Series
Microsystems Seminar Series
Lockheed Martin Robotics Seminar Series
Advanced Networks Colloquia Series
Model-Based Systems Engineering Colloquia Series

Submit an event to the ISR calendar Click here

News links

Current news
Search news
News archives