2024-06-11-ohnishi24a.md

title	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Signatures meet dynamic programming: Generalizing Bellman equations for trajectory following	Path signatures have been proposed as a powerful representation of paths that efficiently captures the path’s analytic and geometric characteristics, having useful algebraic properties including fast concatenation of paths through tensor products. Signatures have recently been widely adopted in machine learning problems for time series analysis. In this work we establish connections between value functions typically used in optimal control and intriguing properties of path signatures. These connections motivate our novel control framework with signature transforms that efficiently generalizes the Bellman equation to the space of trajectories. We analyze the properties and advantages of the framework, termed signature control. In particular, we demonstrate that (i) it can naturally deal with varying/adaptive time steps; (ii) it propagates higher-level information more efficiently than value function updates; (iii) it is robust to dynamical system misspecification over long rollouts. As a specific case of our framework, we devise a model predictive control method for path tracking. This method generalizes integral control, being suitable for problems with unknown disturbances. The proposed algorithms are tested in simulation, with differentiable physics models including typical control and robotics tasks such as point-mass, curve following for an ant model, and a robotic manipulator.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	ohnishi24a	0	Signatures meet dynamic programming: {G}eneralizing {B}ellman equations for trajectory following	466	479	466-479	466	false	Ohnishi, Motoya and Akinola, Iretiayo and Xu, Jie and Mandlekar, Ajay and Ramos, Fabio	given family Motoya Ohnishi given family Iretiayo Akinola given family Jie Xu given family Ajay Mandlekar given family Fabio Ramos	2024-06-11		Proceedings of the 6th Annual Learning for Dynamics & Control Conference	242	inproceedings	date-parts 2024 6 11	https://proceedings.mlr.press/v242/ohnishi24a/ohnishi24a.pdf