Benchmarking recipes (Lauer et al.)

[axel-lauer]

Description

This PR implements a set of benchmarking recipes for comparison of different metrics (RMSE, bias, correlation, EMD) calculated for a given model simulation to the results from an ensemble of (model) datasets:

• model_evaluation/recipe_model_benchmarking_annual_cycle.yml (annual cycle)
• model_evaluation/recipe_model_benchmarking_boxplots.yml (boxplots)
• model_evaluation/recipe_model_benchmarking_diurnal_cycle.yml (diurnal cycle)
• model_evaluation/recipe_model_benchmarking_maps.yml (map plots of 2-dim variables)
• model_evaluation/recipe_model_benchmarking_timeseries.yml (time series)
• model_evaluation/recipe_model_benchmarking_zonal.yml (zonal mean plots of 3-dim variables)

For this, the existing monitoring diagnostics monitoring/monitor.py and monitor/multi_datasets.py have been extended.

The new diurnal cycle plot has also been added to the following existing recipes:

• model_evaluation/recipe_model_evaluation_clouds_cycles.yml
• monitor/recipe_monitor.yml
• monitor/recipe_monitor_with_refs.yml

Documentation for the benchmarking recipes is available in recipes/recipe_benchmarking.rst, the documentation for monitoring and model evaluation have been updated to include the diurnal cycle plots.

Note for testing

The benchmarking recipes require the new preprocessor functions local_solar_time and distance_metric and the extended version of preprocessor resample_hours.

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🛠 This pull request has a descriptive title
• 🛠 Code is written according to the code quality guidelines
• 🛠 Documentation is available
• 🛠 Tests run successfully
• 🛠 The list of authors is up to date
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 All checks below this pull request were successful

New or updated recipe/diagnostic

• 🧪 Recipe runs successfully
• 🧪 Recipe is well documented
• 🧪 Figure(s) and data look as expected from literature
• 🛠 Provenance information has been added

[alistairsellar]

Great work @schlunma and @axel-lauer !

Some minor suggestions and questions from me.

I note also that readthedocs build still failing (so I haven't fully reviewed the documentation), and a couple of style complaints in run_tests.

[alistairsellar]

Does this line ensure that the data has the right dimensions?

[alistairsellar]

Also, I understand that _plot_benchmarking_zonal() expects the cube to have zonal mean applied as pre-processor, rather than being a function that makes and plots zonal mean of 3D data. Also, often zonal means are 1D line plots (i.e. zonal mean of a 2D lat-lon field), so scope for confusion there too. Is required shape of data documented?