Speaker

Kaiyi Ji

Abstract

Recent tremendous successes of deep learning highly rely on the revolutions of a broad range of learning technologies. Many of these technologies, such as hyperparameter optimization,   neural architecture search, and meta-learning, require bilevel optimization structures. With the increasing scale of datasets and training models, large-scale bilevel optimization emerges recently as an exciting but challenging research topic in deep learning. In this talk, I will first discuss some limitations of existing bilevel methods on the scalability with big data and deep neural networks. I will then present two novel algorithms, inspired by Neumann Series  (NS) and Evolution Strategy (ES) in stochastic optimization, for addressing these challenges. I also present the theoretical guarantee of the computational efficiency to support our design principles, where some analysis tools can be of independent interest.

Bio

Dr. Kaiyi Ji is currently a postdoctoral research fellow in the Department of Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor. He received his Ph.D. degree from the Department of Electrical and Computer Engineering at the Ohio State University in 2021. He was also a visiting student research collaborator at the Department of Electrical Engineering at the Princeton University in 2020. Dr. Ji received the B.S. degree from University of Science and Technology of China in 2016. Dr. Ji’s research interests span over bilevel machine learning, meta-learning, large-scale optimization, and learning in complex systems, and his research investigation ranges from theory, algorithms, to applications. Dr. Ji is the recipient of several awards, including the Presidential Fellowship and University Fellowship from OSU. Dr. Ji co-organized the invited session on bilevel machine learning in CISS 2022.

Talk url | Passcode: 766398 [Video-off please during talk]