Provide support in generic worker payload builder for docker worker properties

Open
#289 0 comments 0 reactions 1 assignee View on GitHub

@petemoore is already working on this.

Since May 7, 2026.

Assessment

This issue has not been assessed yet.

Description

Taskgraph has worker-implementation-specific payload builders to convert abstract job definitions into taskcluster tasks. Payload builders declare a schema for the job properties they support. Jobs designed for Docker Worker contain properties that are currently not allowed by the Generic Worker payload builder, such as docker-image, allow-ptrace etc.

The d2g tool demonstrates that it is possible to transform a Docker Worker task definition into an equivalent Generic Worker task definition, and soon Generic Worker will perform this translation internally when running Docker Worker task payloads. However, task definitions designed for Docker Worker will not be able to use features that exist in Generic Worker only. Therefore it would be beneficial for the Generic Worker payload builder to support Docker Worker job definitions natively, and directly generate Generic Worker payloads that do not need to run through d2g inside of Generic Worker.

The d2g source code and command line utility can be used as a reference for understanding how to support Docker Worker features under Generic Worker. Furthermore, to validate that the updated payload builder works as anticipated, tests can be written that generate Docker Worker payloads from job definitions, transform them to Generic Worker using d2g, and compare the results to the same input job processed directly with the Generic Worker payload builder. The two should ideally be identical.

Note, the advantage of doing this, rather than having a transform step that runs d2g after generating a Docker Worker payload, is that eventually we can remove the Docker Worker payload builder altogether, and reduce the code footprint and complexity of dealing with multiple payload formats.

Dominant language
Python
Stars
26
Forks
55
Avg merge
2d 6m
Merged PRs (30d)
15

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from taskcluster/taskgraph

All issues in taskcluster/taskgraph

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.