treeverse / treeverse/dvc

Support Access of Dictionary Items in matrix-stages

Open
#10,094 2 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request p2-medium
Dominant language
Python
Stars
15.9k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

When I work with matrix-stages I would like to use further parameters for each item, specified in the params.yaml
A prototpy-stage would look like

# dvc.yaml
stages:
  eval:
    matrix:
      model: ${models}
      testset: ${testsets}
   cmd: python src/eval_script.py --model ${item.model} --evalset ${item.evalset} --model-family ${item.model.family}
...

And the params.yaml would be:

# params.yaml
models:
  m1:
    family: b21
  m2: 
    family: x22
      
testsets: [evalA, evalB]

This is similar to foreach-stages, where you can access properties by calling ${item.property}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing how matrix-stages expands the model and testset entries from dvc.yaml and params.yaml, then compare that path with foreach-stages' item.property handling. Done means dictionary-valued matrix items support expressions such as ${item.model.family}, with coverage for the example behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, devtools
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.