PIMS-CRG Seminar / UCM Applied Math Colloquium: Melvin Leok
Topic
Geometric adjoint sensitivity analysis with applications to neural ODEs and PDE constrained optimization
Speakers
Details
Abstract: Adjoint systems are widely used to inform control, optimization, and design in systems described by ODEs or DAEs. At the level of PDEs, advection equations and mean field games also have the structure of adjoint systems, and all adjoint systems have a Hamiltonian structure.
We utilize symplectic and presymplectic geometry to investigate the properties of adjoint systems associated with ODEs and DAEs, respectively. We show that the adjoint-variational quadratic conservation laws, which are critical to adjoint sensitivity analysis, arise from (pre)symplecticity of such adjoint systems.
We develop structure-preserving numerical methods for such systems using Galerkin Hamiltonian variational integrators which admit discrete analogues of these quadratic conservation laws. This can be used to train neural networks arising from time-discretizations of neural ODEs, even ones based on implicit-in-time discretization.
Finally, we investigate the geometric structure of adjoint systems associated with evolutionary partial differential equations at the fully continuous, semi-discrete, and fully discrete levels and the relations between these levels. We show that the adjoint system associated with an evolutionary PDE has an infinite-dimensional Hamiltonian structure, which is useful for connecting the fully continuous, semi-discrete, and fully discrete levels.
Additional Information
Biography: Melvin Leok is professor of mathematics and director of the Computational Science, Mathematics and Engineering program at the University of California, San Diego. His research interests are in computational geometric mechanics, computational geometric control theory, discrete differential geometry, and structure-preserving numerical schemes, and particularly how these subjects relate to systems with symmetry. He received his Ph.D. in 2004 from the California Institute of Technology in Control and Dynamical Systems under the direction of Jerrold Marsden. He was a PIMS Marsden Memorial Lecturer, Simons Fellow in Mathematics, three-time NAS Kavli Frontiers of Science Fellow, and has received the DoD Newton Award for Transformative Ideas, NSF Faculty Early Career Development (CAREER) Award, SciCADE New Talent Prize, SIAM Student Paper Prize, Leslie Fox Prize (second prize) in Numerical Analysis, A*STAR International Fellowship, and Loke Cheng-Kim Foundation Scholarship. He has given plenary talks at Foundations of Computational Mathematics, NUMDIFF, and the IFAC Workshop on Lagrangian and Hamiltonian Methods for Nonlinear Control, and is the coauthor of a research monograph entitled, “Global Formulations of Lagrangian and Hamiltonian Dynamics on Manifolds.”