perturb single instance vs. perturb in filter

[hkershaw-brown]

Note copied these comments from issue #533

nancycollins commented 41 minutes ago
this is kind of a tangent, but i'd love to make 'perturb_single_instance' more visible to our users.

when running small models which cycle many time steps inside filter (e.g. L96) i know people find it simpler to include the perturb step inside filter.

but for running large models where you are using a long shell script that runs filter, then runs the model advances, then filter, etc - it seems best to have the first run of filter use exactly the same namelist and files as all the other steps. it makes the shell scripts cleaner and leaves fewer spaces for errors to creep in.

for that reason i'd love to see our scripts use 'perturb_single_instance' to generate the initial ensemble if starting up that way, and use 'fill_inflation_restart' to generate an initial set of inflation files so that the first step can read inflation values from files just like all the other steps.

are there any other differences for the first run of filter which have caused problems in the past that i'm forgetting?

the way this relates to this issue is that if you did run 'perturb_single_instance' first and then run filter, i would think you should see the same files and same values in the 'forecast' output as you'd do if you had filter do the perturb. at least that's my thinking here.

@kdraeder
Member
Author
kdraeder commented 4 minutes ago
On the tangent; I agree that perturb_single_instance would simplify the start of many DART+CAM assims.
It would have the added benefit that the ensemble would have spread before the first hindcast,
so that the first assimilation would be more consistently useful. An example of "not so useful", that I'm working with now, happens because I perturb T in the ensemble after the hindcast and before the assimilation,
but only winds are observed, so there are no increments in any field.
Another difference between cycle 1 and 2 in DART+CAM is changing CESM's CONTINUE_RUN from FALSE
to TRUE. But that could be done at the end of assimilate.csh, where it would be effective
after the first cycle and harmless for all the other cycles.
A remaining reason to stop after 1 cycle is to closely examine the results to see if they are what's intended.
That's not always strictly necessary (snort), and using perturb_single_instance, etc. would make it more tempting to skip this, so this could be an unintended or undesirable consequence.

CMCC may be experiencing the problem of starting an assimilation by telling filter to use inf_initial = 1.0,
but assimilating millions of obs, so that the spread is crushed.

I also made the mistake of linking the initial ensemble of CAM initial files to a single state (to be perturbed),
but not linking the ICs files for the other components (CLM, CICE, MOSART) to the same state.

[nancycollins]

kevin -- good points, although i'd argue that even if you do stop after cycle 1 to inspect things (sounds like a very good idea), if it all looks good you can then just continue without editing anything else. (or forgetting to edit something else!) :)

for the CMCC, it sounds like they might not have enough spread in their initial ensemble if assimilating that many obs crushes the ensemble spreads that much.

maybe they should be creating initial inflation files with a value of 1.1 or something larger if that's what's needed. and if they're not already doing this, i'd recommend they create full inflation files outside of the first run of filter. then they can't make the (common) mistake of forgetting to turn the 'read inflation from file' flag from false to true for subsequent runs.