Unable to complete the notebook "Advanced regridding of data" due to an error during mv.regrid #58
Description
What happened?
The rendering of first few plots of the notebook is fine but I get an error message
when executing the mv.regrid command (cell 13):
fire_1x1_target_grid = mv.regrid(target_grid,
data=fire)
- python advanced_regrid.py
2024-05-16 14:37:03,253 INFO Welcome to the CDS
2024-05-16 14:37:03,253 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/cems-fire-historical
2024-05-16 14:37:03,391 INFO Request is completed
2024-05-16 14:37:03,391 INFO Downloading https://download-0005-clone.copernicus-climate.eu/cache-compute-0005/cache/data5/dataset-cems-fire-historical-0bb8bfc5-9cd8-403c-89da-c38fadc3491a.zip to fire.zip (182.5K)
2024-05-16 14:37:04,138 INFO Download rate 244.4K/s
Python - ERROR - 20240516.143735 - Line 0 in 'PythonScript': concept: no match for paramId=255 [ecCodes]
Python - ERROR - 20240516.143735 - Line 0 in 'PythonScript': concept: input handle edition=2, centre=ecmf [ecCodes]
Python - ERROR - 20240516.143735 - Line 0 in 'PythonScript': Please check the Parameter Database 'https://codes.ecmwf.int/grib/param-db/?id=255' [ecCodes]
Python - ERROR - 20240516.143735 - Line 0 in 'PythonScript': Assertion failed: h in HandleDeleter, line 66 of /home/conda/feedstock_root/build_artifacts/metview_1712857313448/work/mir/src/mir/util/Grib.h
Traceback (most recent call last):
File "/home/jmb001/jmscript/PY/ECMWF/Metview/advanced_regrid.py", line 177, in
fire_1x1_target_grid = mv.regrid(target_grid,
File "/home/jmb001/.conda/envs/ecmwf/lib/python3.10/site-packages/metview/bindings.py", line 1298, in wrapped
return value_from_metview(val)
File "/home/jmb001/.conda/envs/ecmwf/lib/python3.10/site-packages/metview/bindings.py", line 1243, in value_from_metview
raise retval
Exception: Metview error: concept: no match for paramId=255 [ecCodes]
I'm new to Metview so it would be hazardous to offer an explanation. However a few lines in the log (see Relevant Log output below) seems to point towards missing metadata.
NETCDF => (nil) 0
No mapping for NetCDF standard name [] DataInputVariable[name=danger_risk,nc=danger_risk]
No mapping for NetCDF standard name [] DataInputVariable[name=danger_risk,nc=danger_risk]
Regrid - ERROR - 20240515.181123 - concept: no match for paramId=255 [ecCodes]
Regrid - ERROR - 20240515.181123 - concept: input handle edition=2, centre=ecmf [ecCodes]
Regrid - ERROR - 20240515.181123 - Please check the Parameter Database 'https://codes.ecmwf.int/grib/param-db/?id=255' [ecCodes]
grib_util_set_spec: Failed to change edition to 2: Concept no match
Assertion failed: h in HandleDeleter, line 66 of /home/conda/feedstock_root/build_artifacts/metview_1712857313448/work/mir/src/mir/util/Grib.h
The specific input variable (danger_risk) is read from a Netcdf file and might cause
a problem for the mapping to the output (GRIB) during the MIR regridding ?
Am I missing some ancillary datafiles related to grib ?
- Aside: Another detail caught my attention (since I have not installed the source code for Metview)...
this reference to Grib.h include file in a (unknown to me) conda directory
'PythonScript': Assertion failed: h in HandleDeleter, line 66 of /home/conda/feedstock_root/build_artifacts/metview_17
12857313448/work/mir/src/mir/util/Grib.h
What are the steps to reproduce the bug?
I guess run the notebook on the website:
https://metview.readthedocs.io/en/latest/examples/advanced_regrid.html
Version
I've recently install Metview 5.22.0 with conda
Platform (OS and architecture)
cat /etc/os-release --> Ubuntu 22.04.4 LTS uname -a ---> Linux 4.18.0-240.el8.x86_64
Relevant log output
I reran the script after activating debugging information
(export METVIEW_PYTHON_DEBUG=1 ; export MIR_DEBUG=1 ; export MARS_DEBUG=1 )
Here's an excerpt of the listing:
event - INFO - 20240515.181123 - Starting service: regrid
event - INFO - 20240515.181123 - With command : " /home/jmb001/.conda/envs/ecmwf/lib/metview-bundle/bin/Regrid"
event - INFO - 20240515.181123 - Got connection
event - INFO - 20240515.181123 - incoming host is localhost.sack.nb.ec.gc.ca (127.0.0.1)
event - INFO - 20240515.181123 - Register: regrid jmb001 hpcr5-vis5 1431274 as ref 24
event - INFO - 20240515.181123 - regrid : Timeout = 300 Alive = 0, Max = 1
event - INFO - 20240515.181123 - Maximum value for regrid : 1
event - INFO - 20240515.181123 - Got connection
event - INFO - 20240515.181123 - incoming host is localhost.sack.nb.ec.gc.ca (127.0.0.1)
event - INFO - 20240515.181123 - Register: regrid@1431354 jmb001 hpcr5-vis5 1431354 as ref 25
pool - INFO - 20240515.181123 - [regrid@1431354] - Object /Process@1430501/PythonScript, line 0:12 [REGRID] not cached
Regrid::serve in
Dataset: pass1...
Dataset: pass1 done
Dataset: pass2...
Assuming that SimpleInputVariable[name=latitude] is a coordinate variable
Assuming that SimpleInputVariable[name=longitude] is a coordinate variable
Assuming that SimpleInputVariable[name=time] is a coordinate variable
Variable 'danger_risk' has both 'missing_value' and '_FillValue' attributes
Variable 'danger_risk: dimension 'time' not listed in coordinates
Variable 'danger_risk: dimension 'latitude' not listed in coordinates
Variable 'danger_risk: dimension 'longitude' not listed in coordinates
Dataset: pass2...
Dataset: pass4...
Dataset: pass4 done
netcdf /fs/site5/eccc/mrd/rpnenv/jmb001/PY/ECMWF/Metview/ECMWF_FWI_DANGER_RISK_20200515_1200_hr_v3.1_con.nc{
dimensions:
latitude = 721 ;
longitude = 1440 ;
time = 1 ;
variables:
// Kind is data
// Class is DataInputVariable[name=danger_risk,nc=danger_risk]
// Coordinates:
// CoordinateInputVariable[name=time]
// CoordinateInputVariable[name=latitude]
// CoordinateInputVariable[name=longitude]
float danger_risk(time, latitude, longitude) ;
danger_risk:_FillValue = -9999 ;
danger_risk:missing_value = -9999 ;
danger_risk:title = "Danger Risk Class" ;
danger_risk:units = "-" ;
// Kind is coordinate
// Class is CoordinateInputVariable[name=latitude]
float latitude(latitude) ;
latitude:axis = "Y" ;
latitude:long_name = "latitude" ;
latitude:standard_name = "latitude" ;
latitude:units = "degrees_north" ;
// Kind is coordinate
// Class is CoordinateInputVariable[name=longitude]
float longitude(longitude) ;
longitude:axis = "X" ;
longitude:long_name = "longitude" ;
longitude:standard_name = "longitude" ;
longitude:units = "degrees_east" ;
// Kind is coordinate
// Class is CoordinateInputVariable[name=time]
// Codec is GregorianCalendar[reference=2024-05-15 18:11:23, calendar=standard, offset=0]
float time(time) ;
time:axis = "T" ;
time:calendar = "standard" ;
time:standard_name = "time" ;
time:units = "days since 2020-05-15 00:00 UTC" ;
// global attributes:
:CDI = "Climate Data Interface version 1.9.5 (http://mpimet.mpg.de/cdi)" ;
:CDO = "Climate Data Operators version 1.9.5 (http://mpimet.mpg.de/cdo)" ;
:Conventions = "CF-1.6" ;
:ECMWF fire model = "3.1" ;
:License = "Copernicus" ;
:NCO = "netCDF Operators version 4.9.2 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)" ;
:Reference date = "20200515" ;
:cdo_openmp_thread_number = 8 ;
:history = "Wed Jul 29 18:32:39 2020: cdo -P 8 -z zip_1 -mul ECMWF_FWI_DANGER_RISK_20200515_1200_hr_v3.1_con.tmp era5_hr_lsmask0.01.nc ECMWF_FWI_DANGER_RISK_20200515_1200_hr_v3.1_con.nc
Wed Jul 29 18:32:38 2020: cdo -P 8 -smooth,nsmooth=1,maxpoints=10,radius=0.2deg -remapnn,griddes_hr ECMWF_FWI_DANGER_RISK_20200515_1200_hr_v3.1_con.nc ECMWF_FWI_DANGER_RISK_20200515_1200_hr_v3.1_con.tmp
Wed Jul 29 18:32:37 2020: cdo -f nc4 -selvar,danger_risk ECMWF_FWI_20200515_1200_hr.nc ECMWF_FWI_DANGER_RISK_20200515_1200_hr_v3.1_con.nc
Wed Jul 29 18:32:26 2020: ncatted -O -a version,global,c,c,3.1 ECMWF_FWI_20200515_1200_hr.nc
Wed Jul 29 18:32:26 2020: cdo -f nc4 -z zip_1 -selvar,fwi,ffmc,dmc,dc,isi,bui,dsr,danger_risk ecfire.nc ECMWF_FWI_20200515_1200_hr.nc
Wed Jul 29 18:32:24 2020: ncatted -O -a version,global,c,c,3.1 ecfire.nc" ;
:version = "3.1" ;
}
NC Field[variable=DataInputVariable[name=danger_risk,nc=danger_risk]]
Variable::lookupCoordinateInputVariable[name=latitude] has standard_name latitude
Variable::lookupCoordinateInputVariable[name=longitude] has standard_name longitude
GRIDSPEC is RegularLL[bbox=90/0/-90/359.75,grid=0.25/0.25,ni=1440,nj=721]
RegularLL[bbox=90/0/-90/359.75,grid=0.25/0.25,ni=1440,nj=721]
{
"air_pressure_at_sea_level": {
"paramId": number(151)
},
"air_temperature": {
"paramId": number(167)
},
"atmosphere_relative_vorticity": {
"discipline": number(0),
"parameterCategory": number(2),
"parameterNumber": number(12)
},
"cloud_area_fraction_assuming_maximum_random_overlap": {
"paramId": number(164)
},
"dew_point_temperature": {
"paramId": number(168)
},
"divergence_of_wind": {
"discipline": number(0),
"parameterCategory": number(2),
"parameterNumber": number(13)
},
"eastward_wind": {
"discipline": number(0),
"parameterCategory": number(2),
"parameterNumber": number(2)
},
"geopotential": {
"discipline": number(0),
"parameterCategory": number(3),
"parameterNumber": number(4)
},
"geopotential_height": {
"discipline": number(0),
"parameterCategory": number(3),
"parameterNumber": number(5)
},
"kinetic_energy_dissipation_in_atmosphere_boundary_layer": {
"discipline": number(0),
"parameterCategory": number(2),
"parameterNumber": number(20)
},
"lagrangian_tendency_of_air_pressure": {
"discipline": number(0),
"parameterCategory": number(2),
"parameterNumber": number(8)
},
"land_binary_mask": {
"discipline": number(2),
"parameterCategory": number(0),
"parameterNumber": number(0),
"typeOfFirstFixedSurface": number(1)
},
"lwe_thickness_of_surface_snow_amount": {
"paramId": number(141)
},
"moisture_content_of_soil_layer": {
"paramId": number(42)
},
"northward_wind": {
"discipline": number(0),
"parameterCategory": number(2),
"parameterNumber": number(3)
},
"open_sea_surface_temperature": {
"paramId": number(34)
},
"relative_humidity": {
"discipline": number(0),
"parameterCategory": number(1),
"parameterNumber": number(1)
},
"sea_ice_area_fraction": {
"paramId": number(31)
},
"snow_density": {
"paramId": number(33)
},
"soil_temperature": {
"paramId": number(139)
},
"specific_humidity": {
"discipline": number(0),
"parameterCategory": number(1),
"parameterNumber": number(0)
},
"surface_air_pressure": {
"discipline": number(0),
"parameterCategory": number(3),
"parameterNumber": number(0),
"typeOfFirstFixedSurface": number(1)
},
"surface_albedo": {
"paramId": number(243)
},
"surface_downward_eastward_stress": {
"discipline": number(0),
"parameterCategory": number(2),
"parameterNumber": number(38)
},
"surface_downward_northward_stress": {
"discipline": number(0),
"parameterCategory": number(2),
"parameterNumber": number(37)
},
"surface_downwelling_shortwave_flux_in_air": {
"discipline": number(0),
"parameterCategory": number(4),
"parameterNumber": number(7),
"typeOfFirstFixedSurface": number(1),
"typeOfStatisticalProcessing": number(1)
},
"surface_net_downward_shortwave_flux": {
"discipline": number(0),
"parameterCategory": number(4),
"parameterNumber": number(9),
"typeOfFirstFixedSurface": number(1),
"typeOfStatisticalProcessing": number(1)
},
"surface_net_upward_longwave_flux": {
"discipline": number(0),
"parameterCategory": number(5),
"parameterNumber": number(5),
"typeOfFirstFixedSurface": number(1),
"typeOfStatisticalProcessing": number(1)
},
"surface_roughness_length": {
"discipline": number(2),
"parameterCategory": number(0),
"parameterNumber": number(1)
},
"surface_temperature": {
"paramId": number(235)
},
"surface_upward_latent_heat_flux": {
"discipline": number(0),
"parameterCategory": number(0),
"parameterNumber": number(10),
"typeOfFirstFixedSurface": number(1),
"typeOfStatisticalProcessing": number(1)
},
"surface_upward_sensible_heat_flux": {
"discipline": number(0),
"parameterCategory": number(0),
"parameterNumber": number(11),
"typeOfFirstFixedSurface": number(1),
"typeOfStatisticalProcessing": number(1)
},
"toa_outgoing_longwave_flux": {
"discipline": number(0),
"parameterCategory": number(5),
"parameterNumber": number(5),
"typeOfFirstFixedSurface": number(8),
"typeOfStatisticalProcessing": number(1)
},
"wind_speed": {
"paramId": number(49)
},
"x_wind": {
"paramId": number(165)
},
"y_wind": {
"paramId": number(166)
}
}
NETCDF <UNDEFINED> => (nil) 0
No mapping for NetCDF standard name [<UNDEFINED>] DataInputVariable[name=danger_risk,nc=danger_risk]
Regrid - ERROR - 20240515.181123 - concept: no match for paramId=255 [ecCodes]
Regrid - ERROR - 20240515.181123 - concept: input handle edition=2, centre=ecmf [ecCodes]
Regrid - ERROR - 20240515.181123 - Please check the Parameter Database 'https://codes.ecmwf.int/grib/param-db/?id=255' [ecCodes]
grib_util_set_spec: Failed to change edition to 2: Concept no match
Assertion failed: h in HandleDeleter, line 66 of /home/conda/feedstock_root/build_artifacts/metview_1712857313448/work/mir/src/mir/util/Grib.h
backtrace [1] stack has 17 addresses
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libeckit.so+eckit::BackTrace::dump[abi:cxx11]())0x1a9
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libeckit.so+eckit::handle_assert(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, eckit::CodeLocation const&))0x534
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libmir.so+mir::output::GribOutput::save(mir::param::MIRParametrisation const&, mir::context::Context&))0x36df
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libmir.so+mir::action::io::Save::execute(mir::context::Context&) const)0x16f
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libmir.so+mir::action::Action::perform(mir::context::Context&) const)0xb4
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libmir.so+mir::action::ActionPlan::execute(mir::context::Context&) const)0x104
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libmir.so+mir::action::Job::execute(mir::util::MIRStatistics&) const)0x4e
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libmir.so+mir::api::MIRJob::execute(mir::input::MIRInput&, mir::output::MIROutput&, mir::util::MIRStatistics&) const)0xf0
(/home/jmb001/.conda/envs/ecmwf/lib/metview-bundle/bin/Regrid+)0xd32b
(/home/jmb001/.conda/envs/ecmwf/lib/metview-bundle/bin/Regrid+)0xdef0
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libMetview.so+MvService::_call_serve(svcid*, request*))0x1d0
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libmarsclient.so+process_service)0x3e9
(/fs/site5/eccc/mrd/rpnenv/jmb001/micromamba/envs/ecmwf/lib/metview-bundle/bin/../lib/libmarsclient.so+service_run)0x1a
(/home/jmb001/.conda/envs/ecmwf/lib/metview-bundle/bin/Regrid+)0x864f
(/lib/x86_64-linux-gnu/libc.so.6+)0x29d90
(/lib/x86_64-linux-gnu/libc.so.6+__libc_start_main)0x80
(/home/jmb001/.conda/envs/ecmwf/lib/metview-bundle/bin/Regrid+)0x86e5
end of backtrace dump ...
Exception: Assertion failed: h in HandleDeleter, line 66 of /home/conda/feedstock_root/build_artifacts/metview_1712857313448/work/mir/src/mir/util/Grib.h
event - INFO - 20240515.181123 - Closing service regrid@1431354
event - INFO - 20240515.181123 - 0 outstanding replies
Python - ERROR - 20240515.181123 - Line 0 in 'PythonScript': concept: no match for paramId=255 [ecCodes]
Python - ERROR - 20240515.181123 - Line 0 in 'PythonScript': concept: input handle edition=2, centre=ecmf [ecCodes]
Python - ERROR - 20240515.181123 - Line 0 in 'PythonScript': Please check the Parameter Database 'https://codes.ecmwf.int/grib/param-db/?id=255' [ecCodes]
Python - ERROR - 20240515.181123 - Line 0 in 'PythonScript': Assertion failed: h in HandleDeleter, line 66 of /home/conda/feedstock_root/build_artifacts/metview_1712857313448/work/mir/src/mir/util/Grib.h
Traceback (most recent call last):
File "/home/jmb001/jmscript/PY/ECMWF/Metview/advanced_regrid.py", line 177, in <module>
fire_1x1_target_grid = mv.regrid(target_grid,
File "/home/jmb001/.conda/envs/ecmwf/lib/python3.10/site-packages/metview/bindings.py", line 1298, in wrapped
return value_from_metview(val)
File "/home/jmb001/.conda/envs/ecmwf/lib/python3.10/site-packages/metview/bindings.py", line 1243, in value_from_metview
raise retval
Exception: Metview error: concept: no match for paramId=255 [ecCodes]
MetviewInvoker: Closing MetviewAccompanying data
No response
Organisation
Environment and Climate Change Canada