Allow time series tasks to run when history files are not present but cached files are

[cbegeman]

Two changes were needed to allow time series tasks to run when history files are not present but cached files contain the relevant information for those time stamps:

• Return a warning rather than an error if the history files are not present for an analysis task. If the time series output is present in the output file for a time series task, it will catch it at the run stage and successfully use this output. Thus, we do not need to return an error at the setup_and_check stage.

  • Ideally, we would systematically check every time stamp in the output file and the input directory to make sure that we have either the raw or processed data for each time stamp in the user-requested range. I'm leaving this for later work or a to-do item during the Polaris port.

• For the MOC time series, look for data between startDate and endDate. Formerly, the task only looked for cached information corresponding to dates that were present in input history files.

[cbegeman]

@xylar Can you take a look at this when you have the chance? And also let me know if that failing check is an issue I need to address? This is the PR that I think is needed for the HR team and which would go into that shared conda env we discussed.

[cbegeman]

This PR is conceptually related to E3SM-Project/zppy#760 in that we need to cache a few more files with zppy to allow MPAS-A to extend time series even when the history files are removed from disk.

[xylar]

@cbegeman, I think this looks good. I will want to run a few tests once you're ready.

I think the failing CI was probably just a glitch on anaconda.org. I've been seeing a lot of them today.

[cbegeman]

@xylar Please do run whatever tests you have time for. I've been running a few tests with history files and cached files present vs. absent with a LR run on chrysalis but I may have missed conditions that could cause problems.

[xylar]

I think it should be sufficient to run the dev test suite on Chrysalis as a do-no-harm test. Your changes look straightforward so I think your tests of the feature are almost certainly sufficient.

[xylar]

Test Suite

I ran the dev test suite on Chrysalis. Results are here:
https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/xasaydavis/analysis_testing/chrysalis/do-not-require-tseries-files-if-cached-output/

The main vs. control run is still running but everything else looks good and I don't expect any trouble with that one.

[xylar]

Happy to approve once this is out of draft mode.

[cbegeman]

@xylar Thank you! Then I think it's ready for you to create a shared environment on chrysalis from this.

[xylar]

Then I think it's ready for you to create a shared environment on chrysalis from this.

What we did for BlueTip was to merge the PR(s) to develop and then create a tag from develop that was used for the install. The tags are named things like blue_tip_env_2025-04-17. Do you want an hr_env_2025-12-02 tag once this is merged? Then, I would create an environment analogous to bluetip_mpas_analysis called hr_mpas_analysis (or whatever you prefer) and an activation script at:

/lcrc/soft/climate/e3sm-unified/load_hr_mpas_analysis.sh

The environment would be an MPAS-Analysis development environment installed from the tag.

[xylar]

Looks good!

[xylar]

I'll go ahead and merge, and I'll make the tag, environment and load script tomorrow once I get confirmation on what you would like to have it be named.

[cbegeman]

Do you want an hr_env_2025-12-02 tag once this is merged? Then, I would create an environment analogous to bluetip_mpas_analysis called hr_mpas_analysis (or whatever you prefer)

@xylar Thanks for pushing forward on this. Both of those names seem appropriate to me.