Support UK radar data by BernardClaxton · Pull Request #1825 · MetOffice/CSET

BernardClaxton · 2025-11-27T11:55:59Z

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

Documentation has been updated to reflect change.
New code has tests, and affected old tests have been updated.
All tests and CI checks pass.
Ensured the pull request title is descriptive.
Conda lock files have been updated if dependencies have changed.
Attributed any Generative AI, such as GitHub Copilot, used in this PR.
Marked the PR as ready to review.

Sylviabohnenstengel · 2026-01-20T16:31:03Z

for info if the upgrade of you branch with main causes problems then I have created a branch of your branch and merged main in now including the addition of plotting transverse mercator. So you should be able to plot the radar data.
Branch: #1759_import_nimrod_merge_main

Sylviabohnenstengel · 2026-01-20T18:22:28Z

#https://github.com/MetOffice/CSET/tree/1759_import_nimrod_merge_main adds nimrod radar switch into rose gui and enables nimrod switch in flow.cylc for the workflow.

jfrost-mo

src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py is the main file for this change, as the stuff in fetch_nimrod is largely unused.

What is our plan with this? Are we going to try and reuse the RES capability in cube_utils? Or do we not really need it given the existing capabilities of CSET?

The recipe and such looks broadly sensible, though I think we can get away with hard-coding some more for the initial simple recipe.

jfrost-mo · 2026-01-23T11:53:43Z

src/CSET/cset_workflow/flow.cylc

+    [[FETCH_NIMROD]]
+    script = rose task-run -v --app-key=fetch_nimrod
+    execution time limit = PT1H
+        [[[environment]]]
+        ANALYSIS_LENGTH = {{ANALYSIS_LENGTH}}
+


We can make fetch_nimrod inherit from the FETCH_DATA family. Then we don't need a new family, don't need to add it to the graph above, and don't need a new dummy task.

See how the surface obs do it on line ~180.

jfrost-mo · 2026-01-23T11:55:07Z

src/CSET/loaders/spatial_field.py

+        #        for model, field, method in itertools.product(
+        for model, field in itertools.product(
+            models, conf.SURFACE_FIELDS, conf.SPATIAL_SURFACE_FIELD_METHOD
+        ):


I don't think need to loop here, as the field will remain constant for radar, and we don't need to worry about different models either.

So I think this can become something like:

if conf.NIMROD_FIELDS_COMP_HOUR: # Note: You might want to use a different setting. yield RawRecipe( recipe="generic_surface_spatial_plot_sequence_radar.yaml", variables={ "VARNAME": conf.NIMROD_FIELDS_COMP_HOUR }, aggregation=False, )

jfrost-mo · 2026-01-23T12:01:44Z

src/CSET/recipes/surface_fields/generic_surface_spatial_plot_sequence_radar.yaml

+      operator: constraints.combine_constraints
+      varname_constraint:
+        operator: constraints.generate_var_constraint
+        varname: $VARNAME


Could we get away with hardcoding VARNAME? Or do we need it to support different resolution data?

jfrost-mo · 2026-01-23T12:02:17Z

src/CSET/recipes/surface_fields/generic_surface_spatial_plot_sequence_radar.yaml

+#      cell_methods_constraint:
+#        operator: constraints.generate_cell_methods_constraint
+#        cell_methods: []
+#        varname: $VARNAME
+#      level_constraint:
+#        operator: constraints.generate_level_constraint
+#        coordinate: "pressure"
+#        levels: []
+#    subarea_type: $SUBAREA_TYPE
+#    subarea_extent: $SUBAREA_EXTENT
+
+#  - operator: collapse.collapse
+#    coordinate: [time]
+#    method: $METHOD


Suggested change

# cell_methods_constraint:

# operator: constraints.generate_cell_methods_constraint

# cell_methods: []

# varname: $VARNAME

# level_constraint:

# operator: constraints.generate_level_constraint

# coordinate: "pressure"

# levels: []

# subarea_type: $SUBAREA_TYPE

# subarea_extent: $SUBAREA_EXTENT

# - operator: collapse.collapse

# coordinate: [time]

# method: $METHOD

jfrost-mo · 2026-01-23T12:02:50Z

src/CSET/recipes/surface_fields/generic_surface_spatial_plot_sequence_radar.yaml

+#title: $MODEL_NAME $VARNAME $METHOD
+#description: Extracts and plots the $METHOD of $VARNAME from all times in $MODEL_NAME.
+title: $VARNAME


Suggested change

#title: $MODEL_NAME $VARNAME $METHOD

#description: Extracts and plots the $METHOD of $VARNAME from all times in $MODEL_NAME.

title: $VARNAME

title: Radar plot of $VARNAME

Add a little bit more to the title so it is useful.

jfrost-mo · 2026-01-23T12:09:27Z

src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py

I'm a bit confused. Are we implementing the fetching of radar in fetch_fcst, or separately in fetch_nimrod?

jfrost-mo · 2026-01-23T12:16:33Z

src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py

+    """Write the dictionary to retrieve Nimrod obs."""
+    files = {
+        "rainaccum_comp_hour": {
+            "basedir": "/data/users/radar/UKnimrod",


Are these paths too Met Office specific? They might want to be pushed into a per-site file and then moved to the CSET restricted files repository.

jfrost-mo · 2026-01-23T12:17:59Z

src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py

+    print(" Copying Nimrod files to directory: ", cycle_nimrod_dir)
+    for i in filepaths:
+        print(" Grabbing Nimrod file: ", i)
+        shutil.copy(i, cycle_nimrod_dir)


Since they are already on disk, can we just symlink to them to avoid copying data about?

jfrost-mo · 2026-01-23T12:31:15Z

src/CSET/cset_workflow/app/fetch_nimrod/bin/cube_utils.py

This file is unused. Is it only a reference for implementing src/CSET/cset_workflow/app/fetch_nimrod/bin/fetch-nimrod.py? (I presume it comes from the RES?)

jfrost-mo · 2026-01-23T12:49:27Z

src/CSET/cset_workflow/app/fetch_nimrod/bin/fetch-nimrod.py

Ah, so this file just calls the fetch_nimrod function in src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py, and none of the other code is currently used for anything.

github-actions · 2026-01-29T10:53:16Z

BernardClaxton added 15 commits October 8, 2025 11:58

add Nimrod radar page to CSET GUI #1759

09b28e4

put nimrod functions in fetch_data.py to grab radar files #1759

af67bf9

added job in cylc workflow to grab nimrod obs #1759

b7eea4a

defined fetchnimrod task as fetch-nimrod.py #1759

0369efb

put in script fetch-nimrod.py #1759

98fb38b

edits to fetch_data.py #1759

450ef6c

edits to fetch-nimrod.py #1759

d6aaf0a

edits to flow-cylc #1759

97c72ad

edits to meta/rose-meta.conf #1759

68b99a7

fetch-nimrod.py update #1759

518c017

constants.py added #1759

1ce6494

added nimrod colorbar #1759

065afab

added rose-meta colorbar #1759

9fe96cd

adding cube_utils

7e146ed

recipe for nimrod #1759

6963f5d

BernardClaxton self-assigned this Nov 27, 2025

BernardClaxton added the enhancement New feature or request label Nov 27, 2025

BernardClaxton requested a review from jfrost-mo November 27, 2025 11:56

BernardClaxton added 3 commits January 20, 2026 14:43

debugging code added to fetch-nimrod.py at L715 #1759

8d73e22

debugging code added to fetch-nimrod.py at L715 #1759

b76224e

Merge remote-tracking branch 'origin/main' into 1759_import_nimrod

671f904

jfrost-mo changed the title ~~supporting UK radar data~~ Support UK radar data Jan 23, 2026

jfrost-mo reviewed Jan 23, 2026

View reviewed changes

Nimrod fetching code moved into fetch_nimrod.py #1759

a63cefc

tidied the Nimrod radio buttons in GUI #1759

a84e98f

SGallagherMet added the R2O Trials Items of significance to the Met Office Research to Operations Team. label Feb 16, 2026

Conversation

BernardClaxton commented Nov 27, 2025 • edited by jfrost-mo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution checklist

Uh oh!

Sylviabohnenstengel commented Jan 20, 2026

Uh oh!

Sylviabohnenstengel commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jfrost-mo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

BernardClaxton commented Nov 27, 2025 •

edited by jfrost-mo

Loading

Sylviabohnenstengel commented Jan 20, 2026 •

edited

Loading

github-actions bot commented Jan 29, 2026 •

edited

Loading