treeverse / treeverse/dvc

Minimal modifications to params.yaml when running --set-param

Open
#8,720 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A: hydra p2-medium regression
Dominant language
Python
Stars
15.9k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

dvc version: 2.38.1

Hi!

Deep learning guy here.
In our experiments we make use of the configuration files to define the structure of the network.
In particular we have some parameters that are lists where each element is a list of parameters that identify
a particulal sumbodule of the network plus its initialization arguments and its inputs.
This is a quite powerful way to define networks but comes at the cost of having
some parameters with complex values. In order to keep them manageable, we use indentation, inlining
and comments to clarify its structure.
Aside from these, we have other parameters that control other aspects of the network that are usually simple values. These are
the ones we want to change during our batches of experiment to find their optimal values.

Here an example:


activation: "relu"
batch_nomalization: True

network: [ # list of modules : [ name, [*parameters]]
[convolution2d, [16, 2, 2]],
[submodule_x, [32, True, 0.5]],
[submodule_y, [32, False, 1.0, 16, True]],
... 
]

In this case network is the complex parameter whose indentation/inlining is useful, while activation and batch_nomalization are the parameters that are usually explored via batches of experiment.

The problem is that when I try to run an experiment using dvc exp run --set-param activation=swish, the resulting
params.yaml file is something like:


activation: swish
batch_nomalization: True

network: 
- -  convolution2d
- - - 16
- - - 2
- - - 2
- - submodule_x
- - - 32
- - - True
- - - 0.5
- - sumbodule_y
- - - 32
- - - False
- - - 1.0
- - - 16
- - - True
...

The reformatting and dropping of comments makes reading and modifying the network after the fact (while preserving the best set of values for the parameters, discovered during past experiments) more compicated.
What I would prefer is localizing the modifications done to the parameters file only to the keys that actually needs to be changed, thus obtaining (in this case):


activation: swish
batch_nomalization: True

network: [ # list of modules : [ name, [*parameters]]
[convolution2d, [16, 2, 2]],
[submodule_x, [32, True, 0.5]],
[submodule_y, [32, False, 1.0, 16, True]],
...
]

What do you think about it?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the dvc exp run --set-param entry point and the code that updates params.yaml, then reproduce the example using a file with comments and nested lists. Done means changing activation updates only that key while preserving the existing formatting, indentation, and comments under network.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, yaml
Domain
tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.