`mip_convert` failures when using a .pp ancil that contains time stamps
- Dominant language
- Python
- Stars
- 8
- Forks
- 7
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 23
Description
If a pp ancil that contains time stamps is used in a mip convert run (in this case trying to process `slthick_ti-sl-hxy-lnd` using the ancil called "test_input_land_fx_u-dk469.pp" from the functional test for the same variable, this error is thrown:
```
No data available for "1850"; please check run_bounds
Traceback (most recent call last):
File "CDDS/mip_convert/mip_convert/request.py", line 124, in convert
produce_mip_requested_variable(variable_name, stream_id, substream, mip_table, user_config,
File "CDDS/mip_convert/mip_convert/requested_variables.py", line 140, in produce_mip_requested_variable
for time_slice in variable.slices_over(period):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File CDDS/mip_convert/mip_convert/new_variable.py", line 234, in slices_over
sliced_input_variables = self._slice_input_variables(date_time, date_times)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "CDDS/mip_convert/mip_convert/new_variable.py", line 860, in _slice_input_variables
raise RuntimeError('No data available for "{}"; please check run_bounds'.format(date_time_str))
RuntimeError: No data available for "1850"; please check run_bounds
```
Here is a look at the data for that ancil, showing that it has start dates:
```
ppfp test_input_land_fx_u-dk469.pp
/ppfp.py: input: test_input_land_fx_u-dk469.pp
DATE 01/07/2026 TIME 10:58:27
START END
Y M D h m Y M D h m PROC STASH EXP LREC LBLEV LEV HLEV
225701010000 000000000000 0 9 AAAAA 27648 0 0.05 0.0
225701010000 000000000000 0 9 AAAAA 27648 0 0.22 0.0
225701010000 000000000000 0 9 AAAAA 27648 0 0.68 0.0
225701010000 000000000000 0 9 AAAAA 27648 0 2.00 0.0
225701010000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
225701010000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
225701010000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
225701010000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
225701010000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
225701010000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
225701010000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
225701010000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
225701010000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
```
For the purposes of processing this variable, i modified this ancil to zero out the start dates e.g.:
```
START END
Y M D h m Y M D h m PROC STASH EXP LREC LBLEV LEV HLEV
000000000000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
000000000000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
000000000000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
000000000000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
000000000000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
000000000000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
000000000000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
000000000000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
000000000000 000000000000 0 216 AAAAA 27648 0 0.00 0.0
000000000000 000000000000 0 9 AAAAA 27648 0 0.05 0.0
000000000000 000000000000 0 9 AAAAA 27648 0 0.22 0.0
```
This allows the variable to be processed.
What i found a little odd is that the functional test does pass when using the pp ancil containing `START` times.
However, i think it's to do with the fact the request used for that functional test has `run_bounds` that happen to match those in the ancil e.g. ` 'run_bounds': '2257-01-01T00:00:00 2257-02-01T00:00:00',`
I'm assuming that ancils shouldn't be containing time stamps like the one in the functional test.
This issue is to add some kind of handling of pp ancil data if it contains time stamps like this. Though this is low priority, as i think such ancils we accept should have no time stamps like this (correct me if im wrong).
This bug was discovered during issue #918
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in CDDS/mip_convert/mip_convert/new_variable.py, especially _slice_input_variables, then trace its callers in requested_variables.py and request.py. Reproduce the failure with the timestamped test_input_land_fx_u-dk469.pp and compare it with the functional test using matching run_bounds. Done means timestamped pp ancils receive defined handling without the reported missing-data failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100