flatten `deps` to facilitate more flexible templating
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.9k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
The templating feature currently admits little flexibility for stage keys (e.g. deps) that expect lists. As a result, foreach stages cannot be given a variable number of dependencies. Causing lists to be flattened would make templating more powerful. For example, you could give extra dependencies to certain templated stages:
stages:
build:
foreach:
us:
deps:
uk:
deps:
- uk_data
do:
cmd: myscript ${key}
deps:
- myscript
- ${item.deps}
I have implemented the change at https://github.com/itcarroll/dvc/blob/feature/flattened-deps/dvc/dependency/__init__.py#L52, by (non-recursive) list flattening in dvc.dependency.loads_from. I am sure it's not the right way to do it, and probably has unintended consequences, but it demonstrates the concept (and no tests failed).
I don't know how templating responds to missing keys, but not having to include the empty list for us would be even more better.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with dvc/dependency/init.py, especially loads_from and the proposed non-recursive list-flattening logic around the referenced lines. Review how templated foreach stages handle deps, including missing keys, then add focused coverage showing variable dependency lists and confirming existing dependency behavior is preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100