2023-12-02-dalal23a.md

title	section	openreview	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Imitating Task and Motion Planning with Visuomotor Transformers	Poster	QNPuJZyhFE	Imitation learning is a powerful tool for training robot manipulation policies, allowing them to learn from expert demonstrations without manual programming or trial-and-error. However, common methods of data collection, such as human supervision, scale poorly, as they are time-consuming and labor-intensive. In contrast, Task and Motion Planning (TAMP) can autonomously generate large-scale datasets of diverse demonstrations. In this work, we show that the combination of large-scale datasets generated by TAMP supervisors and flexible Transformer models to fit them is a powerful paradigm for robot manipulation. We present a novel imitation learning system called OPTIMUS that trains large-scale visuomotor Transformer policies by imitating a TAMP agent. We conduct a thorough study of the design decisions required to imitate TAMP and demonstrate that OPTIMUS can solve a wide variety of challenging vision-based manipulation tasks with over 70 different objects, ranging from long-horizon pick-and-place tasks, to shelf and articulated object manipulation, achieving $70$ to $80%$ success rates. Video results and code at https://mihdalal.github.io/optimus/	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	dalal23a	0	Imitating Task and Motion Planning with Visuomotor Transformers	2565	2593	2565-2593	2565	false	Dalal, Murtaza and Mandlekar, Ajay and Garrett, Caelan Reed and Handa, Ankur and Salakhutdinov, Ruslan and Fox, Dieter	given family Murtaza Dalal given family Ajay Mandlekar given family Caelan Reed Garrett given family Ankur Handa given family Ruslan Salakhutdinov given family Dieter Fox	2023-12-02		Proceedings of The 7th Conference on Robot Learning	229	inproceedings	date-parts 2023 12 2	https://proceedings.mlr.press/v229/dalal23a/dalal23a.pdf