google-research / google-research/flood-forecasting
[CI / Performance] Optimize integration test suite execution time and add slow markers
- Dominant language
- Python
- Stars
- 343
- Forks
- 87
- Avg merge
- 14d 23m
- Merged PRs (30d)
- 1
Description
### Description
The integration test suite in `test/test_integration_pipeline.py` executes **10 complete neural network training runs** and **6 evaluations** on CPU across a full 366-day calendar year (`01/01/2000` to `31/12/2000`) over 5–8 basins.
Because `.github/workflows/pytest-ci.yml` runs `pytest -m "not gpu"` on every push and PR across the 3-OS runner matrix (Ubuntu, macOS, Windows), this suite runs in its entirety on every CI trigger, taking 3.5–5+ minutes per runner (~15 total
runner-minutes per commit).
### Locations
- `test/test_integration_pipeline.py`:
- `_get_base_config_dict()` (lines 96–101: 366-day train and validation windows)
- All 6 `@pytest.mark.integration` test suites
- `pyproject.toml:42`: Already defines `markers = ["slow: tests taking significant execution time"]`, but integration tests do not use this marker.
### Proposed Optimizations
1. **Shrink the Synthetic Date Window**:
In `_get_base_config_dict`, shorten the date range from 366 days to a 31-day training window and a 15-day evaluation window:
```python
'train_start_date': '01/01/2000',
'train_end_date': '31/01/2000',
'validation_start_date': '01/02/2000',
'validation_end_date': '15/02/2000',
'test_start_date': '01/02/2000',
'test_end_date': '15/02/2000',
This reduces the number of forward/backward batches by ~80% while exercising the exact same model zoo, loss, checkpointing, and evaluation code paths.
2. Apply @pytest.mark.slow:
Decorate long-running end-to-end integration tests with @pytest.mark.slow so rapid local workflows and pre-commit checks can run pytest -m "not slow and not gpu".
Contributor guide
Research direction
Start in test/test_integration_pipeline.py at _get_base_config_dict() and review all six @pytest.mark.integration suites, then check the slow marker in pyproject.toml and the pytest command in .github/workflows/pytest-ci.yml. Shorten the synthetic date windows and mark the long-running tests so the intended pytest selections pass while the integration coverage remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- ci-cd, performance, testing
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100