2023-12-02-mandlekar23b.md

title	section	openreview	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Human-in-the-Loop Task and Motion Planning for Imitation Learning	Poster	G_FEL3OkiR	Imitation learning from human demonstrations can teach robots complex manipulation skills, but is time-consuming and labor intensive. In contrast, Task and Motion Planning (TAMP) systems are automated and excel at solving long-horizon tasks, but they are difficult to apply to contact-rich tasks. In this paper, we present Human-in-the-Loop Task and Motion Planning (HITL-TAMP), a novel system that leverages the benefits of both approaches. The system employs a TAMP-gated control mechanism, which selectively gives and takes control to and from a human teleoperator. This enables the human teleoperator to manage a fleet of robots, maximizing data collection efficiency. The collected human data is then combined with an imitation learning framework to train a TAMP-gated policy, leading to superior performance compared to training on full task demonstrations. We compared HITL-TAMP to a conventional teleoperation system — users gathered more than 3x the number of demos given the same time budget. Furthermore, proficient agents ($75%$+ success) could be trained from just 10 minutes of non-expert teleoperation data. Finally, we collected 2.1K demos with HITL-TAMP across 12 contact-rich, long-horizon tasks and show that the system often produces near-perfect agents. Videos and additional results at https://hitltamp.github.io .	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	mandlekar23b	0	Human-in-the-Loop Task and Motion Planning for Imitation Learning	3030	3060	3030-3060	3030	false	Mandlekar, Ajay and Garrett, Caelan Reed and Xu, Danfei and Fox, Dieter	given family Ajay Mandlekar given family Caelan Reed Garrett given family Danfei Xu given family Dieter Fox	2023-12-02		Proceedings of The 7th Conference on Robot Learning	229	inproceedings	date-parts 2023 12 2	https://proceedings.mlr.press/v229/mandlekar23b/mandlekar23b.pdf