Error in cf.write when writing UM data fields
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 150
- Forks
- 23
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 2
Description
Hi folks. I'm encountering issues again with processing UM data. It's a related task to ticket #817 but a different error so I figured I'd start a new topic.
I can load in a single file (either on a surface level, UM tiles, or pressure levels) with no problem and perform usual cf-python tasks like amending and adding attributes, but when it comes to writing the file I get an unhashable type error as follows:
>>> cf.write(f[0], 'test.nc')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240815/lib/python3.11/site-packages/cfdm/decorators.py", line 171, in verbose_override_wrapper
return method_with_verbose_kwarg(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240815/lib/python3.11/site-packages/cf/read_write/write.py", line 808, in write
netcdf.write(
File "/apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240815/lib/python3.11/site-packages/cfdm/decorators.py", line 171, in verbose_override_wrapper
return method_with_verbose_kwarg(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240815/lib/python3.11/site-packages/cfdm/read_write/netcdf/netcdfwrite.py", line 4911, in write
self._file_io_iteration(
File "/apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240815/lib/python3.11/site-packages/cfdm/read_write/netcdf/netcdfwrite.py", line 5153, in _file_io_iteration
self._write_global_attributes(fields)
File "/apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240815/lib/python3.11/site-packages/cfdm/read_write/netcdf/netcdfwrite.py", line 4289, in _write_global_attributes
force_global = {
^
File "/apps/jasmin/jaspy/miniforge_envs/jaspy3.11/mf3-23.11.0-0/envs/jaspy3.11-mf3-23.11.0-0-v20240815/lib/python3.11/site-packages/cfdm/read_write/netcdf/netcdfwrite.py", line 4292, in <dictcomp>
if len(v) == len(fields) and len(set(v)) == 1
^^^^^^
TypeError: unhashable type: 'dict'
I've managed to process hourly and daily files with this workflow and v similar code with no problem so far. I've also double-checked that all the files I'm currently trying to process here are fully formed and compliant, so that's not the issue (I think this was the main problem in my previous ticket).
I have also tested this behaviour with a MWE, where I load a UM file and then try to write it to netcdf, and the same occurs, i.e.
f = cf.read('/path/to/file/UM_file')
cf.write(f[0], 'test.nc')
Can you help me understand what's going on here? I am able to save the fields in iris without a problem, so it doesn't seem to be packing or underlying file problem, but I'd like to use cf-python to maintain consistency with the other files I've post-processed with this method.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with the UM file and minimal cf.read/cf.write example. Start at cf/read_write/write.py and the cfdm netcdfwrite.py _write_global_attributes path named in the traceback, then determine why the global-attribute values contain an unhashable dict. Done means the UM field writes successfully without this TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100