common-workflow-language / common-workflow-language/cwl-v1.3

dynamic `DockerRequirement.dockerPull`

Open
#45 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Common Workflow Language
Stars
6
Forks
4
PR merge metrics
No merged PRs in 30d

Description

We have gotten this requirement often.

https://cwl.discourse.group/t/dynamic-changing-docker-pull-registry-location-in-dockerrequirements/530
https://cwl.discourse.group/t/using-javascript-in-requirements-such-as-dockerpull/814
(and many other times)

It would also improve the WDL to CWL converter, as WDL workflows often have one or more `string` inputs used to specify the name of a container for some particular step(s).

However some CWL workflow runners, like Arvados, want to know the names of all containers at workflow submission time (to copy them to shared storage)

@tetron suggests the addition of a `{ class: ContainerName, dockerPull: user_provided_value }` or similar. Then a workflow runner can scan the inputs object for `class: ContainerName` (and check the whole workflow DAG for any static `class: ContainerName`s) and handle the copying of those containers ahead of time.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the two linked CWL Discourse discussions and the DockerRequirement.dockerPull behavior described in this issue. Trace how workflow inputs and the workflow DAG are represented, then define the specification change needed for runners to identify dynamic container names before submission.

Written by the indexing model from the issue text.

Assessment

Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.