Direct policy search has achieved great empirical success in reinforcement learning. Many recent studies have revisited its theoretical foundation for continuous control, which reveals elegant nonconvex geometry in various benchmark problems. In this talk, we introduce a new and unified Extended Convex Lifting (ECL) framework to reveal hidden convexity in classical optimal and robust control problems from a modern optimization perspective. Our ECL offers a bridge between nonconvex policy optimization and convex reformulations, enabling convex analysis for nonconvex problems. Despite non-convexity and non-smoothness, the existence of an ECL not only reveals that minimizing the original function is equivalent to a convex problem but also certifies a class of first-order non-degenerate stationary points to be globally optimal. Therefore, no spurious stationarity exists in the set of non-degenerate policies. We believe that the new ECL framework may be of independent interest for analyzing nonconvex problems beyond control. This talk is based on our recent work: arxiv.org/abs/2312.15332, and arxiv.org/abs/2406.04001.

Yang Zheng is an assistant professor in the ECE department at UC San Diego. Yang Zheng received his DPhil (Ph.D.) degree in Engineering Science from the University of Oxford in 2019. He received the B.E. and M.S. degrees from Tsinghua University in 2013 and 2015, respectively.  He was a research associate at Imperial College London and was a postdoctoral scholar in SEAS and CGBC at Harvard University. His research interests include learning, optimization, and control of network systems, and their applications to autonomous vehicles and traffic systems. Dr. Zheng received the 2019 European Ph.D. Award on Control for Complex and Heterogeneous Systems, and the 2022 Best Paper Award in the IEEE Transactions on Control of Network Systems. He was also a recipient of the National Scholarship, Outstanding Graduate at Tsinghua University, and the Clarendon Scholarship at the University of Oxford. Dr. Zheng also won an NSF CAREER Award in 2024, and the 2023 Best Graduate Teacher Award from the ECE department at UC San Diego.