MSLC Data

Overview

This repository contains data and additional figures associated with the Metric Score Landscape Challenge submitted to the WMT Metrics Shared Task in 2023 and 2024.

Structure

The repository contains a directory for each MSLC paper: 2023/ and 2024/, each of which contains the a README file along with the data and figures.

For MSLC24 interactive figures, see: https://nrc-cnrc.github.io/MSLC/2024/interactive/charts.html

Licence

The contents of this repository are released under a CC-BY 4.0 licence.

Citing this work

If you choose to use this data, please cite the relevant paper(s):

@inproceedings{knowles-etal-2024-mslc24,
  title = "{MSLC}24: Further Challenges for Metrics on a Wide Landscape of Translation Quality",
  author = "Knowles, Rebecca  and
      Larkin, Samuel  and
      Lo, Chi-kiu",
  editor = "Haddow, Barry  and
      Kocmi, Tom  and
      Koehn, Philipp  and
      Monz, Christof",
  booktitle = "Proceedings of the Ninth Conference on Machine Translation",
  month = nov,
  year = "2024",
  address = "Miami, Florida, USA",
  publisher = "Association for Computational Linguistics",
  url = "https://aclanthology.org/2024.wmt-1.34",
  pages = "475--491",
}

@inproceedings{lo-etal-2023-metric,
  title = "Metric Score Landscape Challenge ({MSLC}23): Understanding Metrics{'} Performance on a Wider Landscape of Translation Quality",
  author = "Lo, Chi-kiu  and
      Larkin, Samuel  and
      Knowles, Rebecca",
  editor = "Koehn, Philipp  and
      Haddow, Barry  and
      Kocmi, Tom  and
      Monz, Christof",
  booktitle = "Proceedings of the Eighth Conference on Machine Translation",
  month = dec,
  year = "2023",
  address = "Singapore",
  publisher = "Association for Computational Linguistics",
  url = "https://aclanthology.org/2023.wmt-1.65",
  doi = "10.18653/v1/2023.wmt-1.65",
  pages = "776--799",
}