Importer skipping timesteps
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 38/100
Research direction
Start by reproducing the Vortex Importer behavior with InputData.zip, the clipping shapefile, and the documented NetCDF options, then compare the imported timesteps with the source data. Check both the Jython and GUI paths, including the intermittent STAGE IV GRIB case and the consistent NetCDF skips. Done means every expected timestep is written without unintended timestamp shifts.
Written by the indexing model from the issue text.
Description
So I'm having this issue where Vortex Importer is skipping timesteps. When using multiple STAGE IV radar GRIB files, it happens randomly, where sometimes it will skip a timestep or two, and sometimes it will write them all correctly, depending on its mood.
The workaround for that problem is to simply rerun it multiple times until it fills out the DSS. However, I'm also getting an issue where it will consistently skip over certain timesteps, when writing from a NetCDF gridset. This happens both when running from Jython and from the GUI.
EDIT: the consistent skips were fixed by duplicating the time coord var to an aux coor var called valid_time, as described below
The NetCDF's are CF-compliant, and have no missing timesteps. I've opened them up and plotted the timesteps that are not going through Vortex properly, and everything looks fine.
To reproduce
Attached below is one NetCDF and one clipping shapefile.
InputData.zip
Run the importer with the following options. I'm using the latest release but the bug was also happening with an older version before upgrading.


The following timesteps are the ones consistently dropping out. The same happens using other clip shapefiles as well. I'm also getting this surprising behavior where the single hourly timestamp is converted to a start time (D part) 0.5 hr earlier and an end time (E part) 0.5 hr later, so I'm having to timeshift the resulting DSS. I suppose that was the designed behavior, it just seems odd to me.
[Timestamp('2020-10-16 17:30:00'),
Timestamp('2020-10-16 18:30:00'),
Timestamp('2020-10-20 15:30:00'),
Timestamp('2020-10-20 16:30:00'),
Timestamp('2020-10-21 23:30:00'),
Timestamp('2020-10-22 00:30:00'),
Timestamp('2020-10-23 03:30:00'),
Timestamp('2020-10-23 04:30:00'),
Timestamp('2020-10-23 18:30:00'),
Timestamp('2020-10-23 19:30:00')]
- Dominant language
- Java
- Stars
- 37
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from HydrologicEngineeringCenter/Vortex
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
HydrologicEngineeringCenter/Vortex#114 · 14 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
HydrologicEngineeringCenter/Vortex#104 · 10 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 30/100
HydrologicEngineeringCenter/Vortex#103 · 3 comments ·
-
geo not recognized Open
Difficulty 3/5 1-2 days Newbie friendliness 35/100
HydrologicEngineeringCenter/Vortex#92 · 6 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
HydrologicEngineeringCenter/Vortex#90 · 7 comments ·
All issues in HydrologicEngineeringCenter/Vortex
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
bug needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
objectionary/hone-maven-plugin#1061 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
spring-projects/spring-modulith#1895 ·