ga4gh / ga4gh/task-execution-schemas

Consider supporting fallback container image lists in tesExecutor

Open
#249 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
95
Forks
32
PR merge metrics
No merged PRs in 30d

Description

WDL allows task authors to specify an array of container images in the `container` requirement (e.g., `container: ["ghcr.io/org/image:latest", "docker.io/org/image:latest", "*"]`). The idea is that each candidate is tried in order until one pulls successfully, which lets workflows run across environments where a preferred registry might not be reachable.

We're implementing this in [Sprocket](https://github.com/stjude-rust-labs/sprocket/pull/698) right now. For backends where we control the pull (Docker, Apptainer), we iterate through the candidates ourselves. But for TES, we don't have that option—`tesExecutor.image` is a single string, and the pull happens server-side. So we're currently just passing the first candidate and hoping for the best, which defeats the purpose of the fallback list.

The simplest path would be to allow `image` to accept an array of strings in addition to a single string. The server tries each in order and uses the first one it can pull. The server would also need to report which image it actually used—probably as a new `image` field on `tesExecutorLog`, so clients know what ran.

Curious whether this has come up before or if there's appetite for it—right now, TES clients have to pick one and submit, which means either the workflow author has to know which registry is reachable from the TES server or the client has to do something ad hoc.

Contributor guide

Open the contributing guide

Research direction

Review the TES schema definitions for tesExecutor.image and tesExecutorLog, focusing on how image values and execution results are represented. Determine how an ordered list of candidate images and the image actually used should be specified, then confirm that the schema can express both request and response behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.