hydra composition: Workflow discrepancies
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.9k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
When using Hydra Composition for configuring DVC experiments, there are a few discrepancies with respect to the "regular" params workflow. This could cause confusion to existing users of dvc params when migrating to Hydra Composition:
- The latest state is not preserved
Without Hydra Composition, if modifications are done via --set-param and then the experiment is persisted, the next exp run with no arguments will reuse the latest modifications applied.
With Hydra Composition, the next exp run with no arguments will still run the composition and dump to params.yaml, overriding the latest --set-param modifications.
Users would need to manually edit the files in hydra.config_dir and/or the default list values in hydra.config_name in order to reflect the latest modifications via --set-param.
- Source of configuration needs to be tracked separately
Without Hydra Composition, tracking the params is enough. A change in params file would result in a new experiment.
With Hydra Composition, tracking only params.yaml could result in unexpected behavior, as manual modifications to files in hydra.config_dir would not be detected by DVC.
Users would need to also track hydra.config_dir.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Hydra Composition guide and reproduce the described dvc exp run and --set-param workflows using params.yaml, hydra.config_dir, and hydra.config_name. Trace how the latest parameter state and configuration sources are handled; done means both workflows preserve expected changes and detect relevant configuration updates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100