Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update reference files for C96C48_ufs_hybatmDA #1415

Closed
RussTreadon-NOAA opened this issue Dec 14, 2024 · 2 comments · Fixed by #1416
Closed

Update reference files for C96C48_ufs_hybatmDA #1415

RussTreadon-NOAA opened this issue Dec 14, 2024 · 2 comments · Fixed by #1416
Assignees

Comments

@RussTreadon-NOAA
Copy link
Contributor

g-w PR #3145 updated the ufs_model hash. This update alters forecast states used by jobs gdas_atmanlvar and enkfgdas_atmensanlobs in g-w CI C96C48_ufs_hybatmDA. As a result the reference checks in both these jobs fail on Hera.

Changes in the var and lgetkf analyses led to failed checks in downstream jobs gdas_atmanlfv3inc, enkfgdas_atmensanlsol, and enkfgdas_atmensanlfv3inc.

gdas_atmanlvar

 0: OOPS_STATS Run end                                  - Runtime:    909.85 sec,  Memory: total:   723.08 Gb, per task: min =     7.19 Gb, max =     8.51 Gb
 0: Run: Finishing oops::Variational<FV3JEDI, UFO and IODA observations> with status = 0
 0: terminate called after throwing an instance of 'oops::TestReferenceIntegerMismatchError'
 0:   what():  Test reference Integer mismatch @ Line:2
 0: Test Val : 504553
 0: Ref  Val : 504557
 0: Delta    : 4
 0: Tolerance: 0
 0: Test Line: 'CostJo   : Nonlinear Jo(Aircraft) = 6.0586857580910716e+05, nobs = 504553, Jo/n = 1.2008026427533027e+00, err = 2.2437136734248115e+00'
 0: Ref Line : 'CostJo   : Nonlinear Jo(Aircraft) = 6.0606844576066174e+05, nobs = 504557, Jo/n = 1.2011892526724666e+00, err = 2.2437324101760723e+00'
srun: error: h32m11: task 0: Aborted (core dumped)

gdas_atmanlfv3inc

 0: OOPS_STATS Run end                                  - Runtime:     27.19 sec,  Memory: total:    39.74 Gb, per task: min =   398.61 Mb, max =   436.27 Mb
 0: Run: Finishing gdasapp::fv3inc with status = 0
 0: terminate called after throwing an instance of 'oops::TestReferenceFloatMismatchError'
 0:   what():  Test reference Float mismatch @ Line:7
 0: Test Val : 3.1016726684570312e+02
 0: Ref  Val : 3.1315420532226562e+02
 0: Delta    : 2.9869384765625000e+00
 0: Relative tolerance: 3.1166073608398437e-01
 0: Absolute tolerance: 1.0000000000000001e-05
 0: Test Line: 'air_temperature                              | Min:+1.7290489196777344e+02 Max:+3.1016726684570312e+02 RMS:+2.5022379933353739e+02'
 0: Ref Line : 'air_temperature                              | Min:+1.7290422058105469e+02 Max:+3.1315420532226562e+02 RMS:+2.5022388437472281e+02'
srun: error: h35m02: task 0: Aborted (core dumped)
srun: Terminating StepId=3836738.0

enkfgdas_atmensanlobs

0: OOPS_STATS Run end                                  - Runtime:    393.17 sec,  Memory: total:   678.79 Gb, per task: min =     6.78 Gb, max =     8.00 Gb
 0: Run: Finishing oops::LocalEnsembleDA<FV3JEDI, UFO and IODA observations> with status = 0
 0: terminate called after throwing an instance of 'oops::TestReferenceFloatMismatchError'
 0:   what():  Test reference Float mismatch @ Line:10
 0: Test Val : 2.2505994886159897e-02
 0: Ref  Val : 2.2395884618163109e-02
 0: Delta    : 1.1011026799678802e-04
 0: Relative tolerance: 2.2450939752161502e-05
 0: Absolute tolerance: 1.0000000000000001e-05
 0: Test Line: 'water_vapor_mixing_ratio_wrt_moist_air       | Min:+1.6836553484722572e-08 Max:+2.2505994886159897e-02 RMS:+5.0110427820345893e-03'
 0: Ref Line : 'water_vapor_mixing_ratio_wrt_moist_air       | Min:+1.6847316430812498e-08 Max:+2.2395884618163109e-02 RMS:+5.0107896876815721e-03'
srun: error: h32m11: task 0: Aborted (core dumped)
srun: Terminating StepId=3830346.0

enkfgdas_atmensanlsol

 0: OOPS_STATS Run end                                  - Runtime:     51.30 sec,  Memory: total:    81.95 Gb, per task: min =   550.30 Mb, max =  1471.40 Mb
 0: Run: Finishing oops::LocalEnsembleDA<FV3JEDI, UFO and IODA observations> with status = 0
 0: terminate called after throwing an instance of 'oops::TestReferenceFloatMismatchError'
 0:   what():  Test reference Float mismatch @ Line:10
 0: Test Val : 2.2505994886159897e-02
 0: Ref  Val : 2.2395884618163109e-02
 0: Delta    : 1.1011026799678802e-04
 0: Relative tolerance: 2.2450939752161502e-05
 0: Absolute tolerance: 1.0000000000000001e-05
 0: Test Line: 'water_vapor_mixing_ratio_wrt_moist_air       | Min:+1.6836553484722572e-08 Max:+2.2505994886159897e-02 RMS:+5.0110427820345893e-03'
 0: Ref Line : 'water_vapor_mixing_ratio_wrt_moist_air       | Min:+1.6847316430812498e-08 Max:+2.2395884618163109e-02 RMS:+5.0107896876815721e-03'
srun: error: h36m19: task 0: Aborted (core dumped)

enkfgdas_atmensanlfv3inc

 0: OOPS_STATS Run end                                  - Runtime:     57.87 sec,  Memory: total:    36.12 Gb, per task: min =   356.64 Mb, max =   391.59 Mb
 0: Run: Finishing gdasapp::fv3inc with status = 0
 0: terminate called after throwing an instance of 'oops::TestReferenceFloatMismatchError'
 0:   what():  Test reference Float mismatch @ Line:9
 0: Test Val : 2.2505994886159897e-02
 0: Ref  Val : 2.2395884618163109e-02
 0: Delta    : 1.1011026799678802e-04
 0: Relative tolerance: 2.2450939752161502e-05
 0: Absolute tolerance: 1.0000000000000001e-05
 0: Test Line: 'water_vapor_mixing_ratio_wrt_moist_air       | Min:+1.6836553484722572e-08 Max:+2.2505994886159897e-02 RMS:+5.0110427820345893e-03'
 0: Ref Line : 'water_vapor_mixing_ratio_wrt_moist_air       | Min:+1.6847316430812498e-08 Max:+2.2395884618163109e-02 RMS:+5.0107896876815721e-03'
srun: error: h33m24: task 0: Aborted (core dumped)
srun: Terminating StepId=3837288.0

This issue is opened to document the updating of reference files

test/testreference/C96C48_ufs_hybatmDA_3dvar-fv3inc.ref
test/testreference/C96C48_ufs_hybatmDA_3dvar.ref
test/testreference/C96C48_ufs_hybatmDA_lgetkf-fv3inc.ref
test/testreference/C96C48_ufs_hybatmDA_lgetkf_observer.ref
test/testreference/C96C48_ufs_hybatmDA_lgetkf_solver.ref
@RussTreadon-NOAA RussTreadon-NOAA self-assigned this Dec 14, 2024
@RussTreadon-NOAA
Copy link
Contributor Author

Work for this issue will be done in feature/update_refs

@RussTreadon-NOAA
Copy link
Contributor Author

Note: The changes in this issue are required for g-w PR #3163 to successfully run g-w CI C96C48_ufs_hybatmDA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant