2023-12-02-margolis23a.md

title	section	openreview	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Learning to See Physical Properties with Active Sensing Motor Policies	Poster	RQ_7yVV8vA	To plan efficient robot locomotion, we must use the information about a terrain’s physics that can be inferred from color images. To this end, we train a visual perception module that predicts terrain properties using labels from a small amount of real-world proprioceptive locomotion. To ensure label precision, we introduce Active Sensing Motor Policies (ASMP). These policies are trained to prefer motor skills that facilitate accurately estimating the environment’s physics, like swiping a foot to observe friction. The estimated labels supervise a vision model that infers physical properties directly from color images and can be reused for different tasks. Leveraging a pretrained vision backbone, we demonstrate robust generalization in image space, enabling path planning from overhead imagery despite using only ground camera images for training.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	margolis23a	0	Learning to See Physical Properties with Active Sensing Motor Policies	2537	2548	2537-2548	2537	false	Margolis, Gabriel B. and Fu, Xiang and Ji, Yandong and Agrawal, Pulkit	given family Gabriel B. Margolis given family Xiang Fu given family Yandong Ji given family Pulkit Agrawal	2023-12-02		Proceedings of The 7th Conference on Robot Learning	229	inproceedings	date-parts 2023 12 2	https://proceedings.mlr.press/v229/margolis23a/margolis23a.pdf