Mechanism for automatically fetching secrets

Open
#13 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
ci-cd, devops, security

Research direction

Start by reading the task definition schema and the existing fetches handling in run_task.py, then inspect the run-task script that executes tasks. Define how the top-level secrets entries are transformed and consumed for environment variables and files, and verify that both example forms work without wrapper scripts.

Written by the indexing model from the issue text.

Description

feature

When a task needs to use a secret, we need to create a custom wrapper script that first downloads the secret from the secrets service, sets it up, and then invokes the regular command. This makes creating tasks that need secrets inconvenient.. There's no way to just "declare" that the secret is needed in the task definition and have everything just work.

I propose we introduce a new run-task based mechanism for handling secrets. At a high level it would work like this:

  1. A task declares which secrets it needs in a top-level secrets key.
  2. The run_task.py transforms would massage this into a format we can easily stuff into an environment variable.
  3. The run-task script reads said env and fetches the required secrets before proceeding with the task

There is precedent here as this is exactly how fetches work.

To start, the schema for defining the secrets should support both environment variables, as well as files. Maybe something like:

secrets:
    - secret: myproject/secret
      key: api_token
      env: API_TOKEN
    - secret: myproject/other/secret
      file: /builds/worker/secret

The above definition would:
A) store the value of api_token in the myproject/secret secret into the API_TOKEN env
B) write the entirety of the myproject/other/secret secret wholesale into a file at /builds/worker/secret

Dominant language
Python
Stars
26
Forks
55
Avg merge
2d 6m
Merged PRs (30d)
15

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from taskcluster/taskgraph

All issues in taskcluster/taskgraph

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.