2022-06-28-baevski22a.md

title	booktitle	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language	Proceedings of the 39th International Conference on Machine Learning	While the general idea of self-supervised learning is identical across modalities, the actual algorithms and objectives differ widely because they were developed with a single modality in mind. To get us closer to general self-supervised learning, we present data2vec, a framework that uses the same learning method for either speech, NLP or computer vision. The core idea is to predict latent representations of the full input data based on a masked view of the input in a self-distillation setup using a standard Transformer architecture. Instead of predicting modality-specific targets such as words, visual tokens or units of human speech which are local in nature, data2vec predicts contextualized latent representations that contain information from the entire input. Experiments on the major benchmarks of speech recognition, image classification, and natural language understanding demonstrate a new state of the art or competitive performance to predominant approaches.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	baevski22a	0	data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language	1298	1312	1298-1312	1298	false	Baevski, Alexei and Hsu, Wei-Ning and Xu, Qiantong and Babu, Arun and Gu, Jiatao and Auli, Michael	given family Alexei Baevski given family Wei-Ning Hsu given family Qiantong Xu given family Arun Babu given family Jiatao Gu given family Michael Auli	2022-06-28		Proceedings of the 39th International Conference on Machine Learning	162	inproceedings	date-parts 2022 6 28	https://proceedings.mlr.press/v162/baevski22a/baevski22a.pdf