Allow fetch tasks to be generated by embedded entries in other tasks
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- build-system, ci-cd
Research direction
Review the two linked prototype comparisons and the TransformSequence discussion, including fetch_only_job_transfoms. Clarify the desired embedded-fetch behavior and formalize a complete approach; done means the behavior is no longer hardcoded and transform generation is defined well enough to support the stated examples.
Written by the indexing model from the issue text.
Description
For simple use cases, embedded fetch definitions into tasks could be much nicer than the indirection of having a separate kind for them. It can also be nice to embed them where only one entry in a kind needs them -- it puts the fetch with the thing that actually needs it, which makes it more obvious what's going on, and lowers maintenance burden a small amount.
I've put together a prototype for how this could work in these patches:
- https://github.com/taskcluster/taskgraph/compare/main...bhearsum:taskgraph:fetches-in-tasks?expand=1
- https://github.com/mozilla-mobile/mozilla-vpn-client/compare/main...bhearsum:mozilla-vpn-client:embedded-fetch?expand=1
It's a bit hacky at the moment, but it does prove that we can generate unrelated tasks (eg: a fetch) while processing tasks from another kind (eg: build). This may seem very strange, and it certainly breaks new ground in taskgraph. When I look at things through the eyes of someone building a project (in this case MozillaVPN), the tasks I really care about are build, test, etc. -- things that produce artifacts that are of value to developers or users. fetch, on the other hand, is an implementation detail. There are certainly good reasons for them existing in a separate kind at times (most obviously, when a single fetch is used by many downstream tasks) -- but that shouldn't be strictly necessary.
Obviously the linked patches are not remotely landable. There's hardcoding that has to be fixed, and most importantly, the generation of the TransformsSequence would need to be formalized (especially the fetch_only_job_transfoms part, where it's taking only part of the job transforms -- I have some ideas on how to clean that up).
We can also just close this as undesirable if it's going too far or a bad idea for some reason.
- Dominant language
- Python
- Stars
- 26
- Forks
- 55
- Avg merge
- 2d 6m
- Merged PRs (30d)
- 15
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from taskcluster/taskgraph
-
Dependency Dashboard Open
Difficulty 5/5 Over a week Newbie friendliness 20/100
taskcluster/taskgraph#952 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
taskcluster/taskgraph#915 ·
-
feature
Difficulty 3/5 1-2 days Newbie friendliness 35/100
taskcluster/taskgraph#911 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
taskcluster/taskgraph#882 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
taskcluster/taskgraph#865 · 1 comment ·
All issues in taskcluster/taskgraph
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
-
🐛 Bug 🔔 Pending processing
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
jumpserver/jumpserver#17584 ·