2023-12-02-goyal23a.md

title	section	openreview	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
RVT: Robotic View Transformer for 3D Object Manipulation	Oral	0hPkttoGAf	For 3D object manipulation, methods that build an explicit 3D representation perform better than those relying only on camera images. But using explicit 3D representations like voxels comes at large computing cost, adversely affecting scalability. In this work, we propose RVT, a multi-view transformer for 3D manipulation that is both scalable and accurate. Some key features of RVT are an attention mechanism to aggregate information across views and re-rendering of the camera input from virtual views around the robot workspace. In simulations, we find that a single RVT model works well across 18 RLBench tasks with 249 task variations, achieving $26%$ higher relative success than the existing state-of-the-art method (PerAct). It also trains 36X faster than PerAct for achieving the same performance and achieves 2.3X the inference speed of PerAct. Further, RVT can perform a variety of manipulation tasks in the real world with just a few ($\sim$10) demonstrations per task. Visual results, code, and trained model are provided at: https://robotic-view-transformer.github.io/.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	goyal23a	0	RVT: Robotic View Transformer for 3D Object Manipulation	694	710	694-710	694	false	Goyal, Ankit and Xu, Jie and Guo, Yijie and Blukis, Valts and Chao, Yu-Wei and Fox, Dieter	given family Ankit Goyal given family Jie Xu given family Yijie Guo given family Valts Blukis given family Yu-Wei Chao given family Dieter Fox	2023-12-02		Proceedings of The 7th Conference on Robot Learning	229	inproceedings	date-parts 2023 12 2	https://proceedings.mlr.press/v229/goyal23a/goyal23a.pdf