MetOffice / MetOffice/CSET

Inconsistent merge behaviour with time coord in read_cubes and filter_cubes

Open
#2,345 2 comments 0 reactions 1 assignee Claimed by @jfrost-mo View on GitHub
documentation
Dominant language
Python
Stars
33
Forks
19
Avg merge
1d 22h
Merged PRs (30d)
30

Description

### Describe the bug
I have files at different time slices which have been pre-processed so that time has been added as a scalar coordinate. I have this data stored in two separate folders, one for each day to load. When trying to load this data into CSET and filter down to a single cube, the choice of input wildcard affects whether the cubelist merging is successful or not.

### How to reproduce
Using the following pattern matching allows `filter_cubes` to successfully merge the cubelist:
```
- operator: read.read_cubes
file_paths: .../2022072[34]/*.nc
```

However, when these paths are listed explicitly in the `read_cubes` operator, `filter_cubes` is unable to merge the cubelist:
```
- operator: read.read_cubes
file_paths: .
- .../20220723/*.nc
- .../20220724/*.nc
```

This appears to be a problem with iris itself not liking time as a scalar coordinate, and in fact I was unable to merge the data into a single cube when just using iris outside of CSET. This was solved by using `iris.util.unify_time_units(cubes)` which is included in `read.read_cubes` operator.

Therefore, it seems likely that this inconsistency is being caused by CSET being unable unify time units over multiple cubelists. Could this be added as a step in `filter_cubes` without breaking existing functionality?

### Expected behaviour
Loading data with time as a scalar coordinate should not depend on whether files are loaded as a single wildcard or from a list of input paths.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.