google-research / google-research/flood-forecasting

[CI / Performance] Optimize integration test suite execution time and add slow markers

Closed
#282 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
343
Forks
87
Avg merge
14d 23m
Merged PRs (30d)
1

Description

### Description
The integration test suite in `test/test_integration_pipeline.py` executes **10 complete neural network training runs** and **6 evaluations** on CPU across a full 366-day calendar year (`01/01/2000` to `31/12/2000`) over 5–8 basins.

Because `.github/workflows/pytest-ci.yml` runs `pytest -m "not gpu"` on every push and PR across the 3-OS runner matrix (Ubuntu, macOS, Windows), this suite runs in its entirety on every CI trigger, taking 3.5–5+ minutes per runner (~15 total
runner-minutes per commit).

### Locations
- `test/test_integration_pipeline.py`:
- `_get_base_config_dict()` (lines 96–101: 366-day train and validation windows)
- All 6 `@pytest.mark.integration` test suites
- `pyproject.toml:42`: Already defines `markers = ["slow: tests taking significant execution time"]`, but integration tests do not use this marker.

### Proposed Optimizations
1. **Shrink the Synthetic Date Window**:
In `_get_base_config_dict`, shorten the date range from 366 days to a 31-day training window and a 15-day evaluation window:
```python
'train_start_date': '01/01/2000',
'train_end_date': '31/01/2000',
'validation_start_date': '01/02/2000',
'validation_end_date': '15/02/2000',
'test_start_date': '01/02/2000',
'test_end_date': '15/02/2000',

This reduces the number of forward/backward batches by ~80% while exercising the exact same model zoo, loss, checkpointing, and evaluation code paths.

2. Apply @pytest.mark.slow:
Decorate long-running end-to-end integration tests with @pytest.mark.slow so rapid local workflows and pre-commit checks can run pytest -m "not slow and not gpu".

Contributor guide

Open the contributing guide

Research direction

Start in test/test_integration_pipeline.py at _get_base_config_dict() and review all six @pytest.mark.integration suites, then check the slow marker in pyproject.toml and the pytest command in .github/workflows/pytest-ci.yml. Shorten the synthetic date windows and mark the long-running tests so the intended pytest selections pass while the integration coverage remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, performance, testing
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.