Atm history files incorrectly written with new compression options

[CatherineThomas-NOAA]

What is wrong?

When using config.ufs parameter quantize_nsd=5, the fields in the atmospheric history files seem to be incorrect. Many fields such as temperature are integers only or for the case of wind rounded to 0.25. Here are two history files with identical settings except for quantize_nsd at C384:

This example was run on Hera for an ATM-only C384/C192 DA experiment. It was confirmed for two cycles of forecasts: first cold start forecast and second warm start forecast.

This behavior has also been confirmed in a C1152 coupled free forecast on Dogwood by @RuiyuSun.

What should have happened?

Could someone from the modeling team confirm what the expected behavior of the compression options should be? Should we see identical history file output? @junwang-noaa @DusanJovic-NOAA @LarissaReames-NOAA

From config.ufs:

  "C384" | "C768" | "C1152" | "C3072")
    zstandard_level=0
    ideflate=1
    quantize_nsd=5

What machines are impacted?

All or N/A

What global-workflow hash are you using?

This was noticed after PR #2914 when C384 was added to the compression option list

Steps to reproduce

1. Clone/build g-w.
2. Run the forecast (coupled or ATM-only) with default compression options for C384 or higher.
3. Ncdump/ncview temperature for the atmos history files.

Additional information

Since the DA uses the history files, this difference shows up very clearly in cycling skill. Fits shown for the first model forecast:

Do you have a proposed solution?

No response

[DusanJovic-NOAA]

I noticed the same issue while looking at some other issues with global workflow high resolution runs. See this comment ufs-community/ufs-weather-model#2439 (comment)

[CatherineThomas-NOAA]

Thanks for pointing me to that discussion @DusanJovic-NOAA. I also see a comment from @junwang-noaa:

the quantize_nsd and quantize_bitround configurations are corresponding to the previous nbits=14 with our customized lossy compression code. The physics group evaluated with results for nbits setting from (nbits=12-32), and decided the nbits=14 to be used in GFSv16. The quantize_nsd=5 is corresponding to nbits=14

This leads me to believe that the expected behavior of these new options should be the same as the previous options, but this is not what we're seeing.

Maybe we should confirm first that all needed options are set properly? What options need to be looked at?

[junwang-noaa]

@DusanJovic-NOAA Is it possible that we can retrieve the previous customize quantization code to see we still see the issue? We do not see this issue with GFSv16 using nbits=14.

[DusanJovic-NOAA]

The meaning of quantize_nsd parameter depends on the value of quantize_mode. When quantize_mode is set to quantize_bitround the value of quantize_nsd determines the number of significant binary digits (ie. bits), and it should be the same as previous nbits, 14 in this case.

See: https://docs.unidata.ucar.edu/netcdf-c/current/group__variables.html#ga669e4b23b9b7fb321b995982972cdd93

This is very confusing because the same parameter (nsd) has different meaning (either decimal or binary) depending on the value of another parameter.

[CatherineThomas-NOAA]

Thanks @DusanJovic-NOAA, so quantize_mode matters. It looks like quantize_mode of bit grooming or granular bit round are decimal, while bit round is binary.

Is the solution then to use a different quantize_mode? Was bit round the intended parameter or are we not setting something properly in the workflow?

[DusanJovic-NOAA]

You can leave quantize_mode to be quantize_bitround just set quantize_nsd to 14.

[CatherineThomas-NOAA]

I ran a quick forecast on Hera with quantize_nsd=14 here : /scratch1/NCEPDEV/stmp2/Catherine.Thomas/ROTDIRS/taper_test9/gdas.20230315/00/model/atmos/history/

The differences between this and taper_test4 (quantize_nsd=0) appear to be noise. Differences in temperature are less than 0.01 K. Differences in winds are less than 1e-4 m/s.

As for file size comparisons:
quantize_nsd=0
-rw-r--r-- 1 Catherine.Thomas stmp 9.0G Nov 26 17:48 gdas.t00z.atmf006.nc

quantize_nsd=14
-rw-r--r-- 1 Catherine.Thomas stmp 2.2G Dec 2 14:19 gdas.t00z.atmf006.nc

[CatherineThomas-NOAA]

Reopening as I missed that this setting is also in config.ufs for the GEFS as well. Pinging @bingfu-NOAA for awareness.

[CatherineThomas-NOAA]

This issue can be closed again since the change was added to the GEFS config files as well (#3184).