common-workflow-language / common-workflow-language/cwltool

Intermediate results are are wrongly cached in case of nested workflow

Open
#696 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

I'm trying to run a workflow multiple times, but with one variable changed. For that, I've created a nested workflow. Repo can be found here:

https://github.com/gijzelaerr/spiel/blob/57c9f7873cb64a822f03a23026cc57e5168cf6e4/multi.cwl

When I run this workflow the cached intermediate results get used, also when the input variable (random_seed) changes. The result is that all runs have the same output. This makes the pipeline very fast, but wrong ;)

Contributor guide

Open the contributing guide

Research direction

Start with the linked multi.cwl reproduction and run the nested workflow multiple times while changing random_seed. Trace how intermediate results are cached between those runs. Done means changing random_seed produces distinct outputs rather than reusing the prior cached results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.