google-research / google-research/flood-forecasting
Design target-free inference without observation files
- Dominant language
- Python
- Stars
- 343
- Forks
- 87
- Avg merge
- 14d 23m
- Merged PRs (30d)
- 1
Description
## Design question
Should `run infer` support predictor-complete basins that have no target/observation file at all?
The current MultiMet loading path retains configured `target_variables` in inference mode and `_load_data()` unconditionally calls `_load_target_features()`. A predictor-complete synthetic inference fixture therefore fails with `FileNotFoundError` before prediction when the target tree is absent.
I am treating this as a design gap/enhancement rather than a confirmed loader regression: the current documentation does not clearly promise target-file-free inference, and supporting it affects more than dataset loading.
## Proposed behavior for discussion
1. Evaluation and fine-tuning continue to require targets.
2. Inference can omit target loading when observation files are unavailable.
3. Inference with supplied targets retains observations and current diagnostic outputs.
4. Sample validation does not require target values in target-free mode.
5. Losses and observation-dependent metrics are skipped when targets are absent.
6. Prediction coordinates and serialization remain identical in both modes.
## Questions
- Should this be automatic when target files are missing, or controlled by an explicit configuration flag?
- Should `target_variables` remain configured to preserve checkpoint/model output semantics even when no target data is loaded?
- What tester/result schema should represent unavailable observations, losses, and metrics?
- Are all-missing placeholder target files the intended current workflow instead?
If the intended behavior is confirmed, I can prepare a separate integration-tested PR covering the dataset, sample validation, tester outputs, and metrics. I would keep it independent from the CSV schema fixes in #265 and #266.
Contributor guide
Research direction
Start with the MultiMet loading path, especially _load_data() and _load_target_features(), then trace dataset loading, sample validation, tester outputs, and metrics used by run infer. Resolve whether target-free inference is supported and how missing observations, losses, and metrics should be represented. Done requires an agreed design and integration tests covering inference with and without targets while preserving prediction coordinates and serialization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100