common-workflow-language / common-workflow-language/cwltool

`$` escape behavior change on a second call of the same command line tool within workflow

Open
#2,217 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

## Summary
the issue was tested with cwltool 3.1.20240112164112 and 3.1.20251031082601 and CWLv1.0

The bug happens when in a CWLv1.0 workflow with several steps that call the same CommandLineTool. This tool runs a script described in `InitialWorkDirRequirement`. This script has variables with escaped `\$`. I've found by `cwltool --debug` that the first step using the tool makes a normal BASH script with `$` variables, but the second step creates escaped `\\$` variables.

## Investigation
It seems that the reason is in the version control of the escaping handling in the `update.py`. The `ORIGINAL_CWLVERSION` is used in the first run of the tool, but the second run of the tool uses the `INTERNAL_VERSION`.

## The toy example.

### Workflow:
```
class: Workflow
cwlVersion: v1.0

requirements:
StepInputExpressionRequirement: {}

inputs:
file1:
label: FASTQ files 1
type: File

file2:
label: FASTQ file 2
type: File

file3:
label: FASTQ file 3
type: File

steps:
cat_1:
run: cat.cwl
in:
file1: file1
file2: file2
output_fn:
valueFrom: "1.txt"
out:
- out_file

cat_2:
run: cat.cwl
in:
file1: cat_1/out_file
file2: file3
output_fn:
valueFrom: "2.txt"
out:
- out_file

outputs:
wf_out:
type: File
outputSource: cat_2/out_file
```

### cat.cwl tool
```
cwlVersion: v1.0
class: CommandLineTool

label: Concatenate files
doc: Concatenate files using `cat`

requirements:
DockerRequirement:
dockerPull: ubuntu:20.04
InitialWorkDirRequirement:
listing:
- entryname: wrapper.sh
entry: |
#!/usr/bin/env bash
FILE1=\$1
FILE2=\$2
OUTFILE=\$3
cat \$FILE1 \$FILE2 > $OUTFILE

baseCommand: ["bash", "wrapper.sh"]

inputs:
file1:
type: File
label: Input file 1
inputBinding:
position: 1

file2:
type: File
label: Input file 1
inputBinding:
position: 2

output_fn:
type: string
label: Output filename
inputBinding:
position: 3

outputs:
out_file:
type: File
outputBinding:
glob: $(inputs.output_fn)
```

The output:
```
cat: '$FILE1': No such file or directory
cat: '$FILE2': No such file or directory
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.