common-workflow-language / common-workflow-language/cwltool
Inputs containing both directories and files within directories fail to get mounted to correct places for CWL >= 1.1 workflows
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
If you pass in a directory and a file contained within this directory to a CWL tool using cwl>=1.1, the file contained
within this directory does not get put into the correct location for the tool to run.
## Expected Behavior
The file contained within the directory should get put in the correct location.
## Actual Behavior
The file contained within the directory does not get put in the correct location when you use cwl >= 1.1.
## Workflow Code
`tool.cwl`
```
cwlVersion: v1.1
class: CommandLineTool
id: tool
baseCommand:
- ls
inputs:
file:
type: File
inputBinding:
position: 1
directory:
type: Directory
inputBinding:
position: 2
outputs: []
```
`inputs.yaml`
```
directory:
class: Directory
path: directory
file:
class: File
path: directory/file
```
## Full Traceback
```pytb
$ cwltool --debug tool.cwl inputs.yaml
INFO /home/user/Documents/checkouts/cwltool/venv/bin/cwltool 3.1.20221115160822
INFO Resolved 'tool.cwl' to 'file:///home/user/Documents/checkouts/cwltool/my/issue2/tool.cwl'
DEBUG [job tool] initializing from file:///home/user/Documents/checkouts/cwltool/my/issue2/tool.cwl#tool
DEBUG [job tool] {
"directory": {
"class": "Directory",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory",
"basename": "directory"
},
"file": {
"class": "File",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory/file",
"size": 0,
"basename": "file",
"nameroot": "file",
"nameext": ""
}
}
DEBUG [job tool] path mappings is {
"file:///home/user/Documents/checkouts/cwltool/my/issue2/directory": [
"/home/user/Documents/checkouts/cwltool/my/issue2/directory",
"/tmp/qu6q87wp/stgdcb1db8c-5f78-4354-b36e-915ec91ebab8/directory",
"Directory",
true
],
"file:///home/user/Documents/checkouts/cwltool/my/issue2/directory/file": [
"/home/user/Documents/checkouts/cwltool/my/issue2/directory/file",
"/tmp/qu6q87wp/stg0fec546f-112c-4c2c-afa0-17256f9bd17a/file",
"File",
true
]
}
DEBUG [job tool] command line bindings is [
{
"position": [
-1000000,
0
],
"datum": "ls"
},
{
"position": [
1,
"file"
],
"datum": {
"class": "File",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory/file",
"size": 0,
"basename": "file",
"nameroot": "file",
"nameext": "",
"path": "/tmp/qu6q87wp/stg0fec546f-112c-4c2c-afa0-17256f9bd17a/file",
"dirname": "/tmp/qu6q87wp/stg0fec546f-112c-4c2c-afa0-17256f9bd17a"
}
},
{
"position": [
2,
"directory"
],
"datum": {
"class": "Directory",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory",
"basename": "directory",
"path": "/tmp/qu6q87wp/stgdcb1db8c-5f78-4354-b36e-915ec91ebab8/directory",
"dirname": "/tmp/qu6q87wp/stgdcb1db8c-5f78-4354-b36e-915ec91ebab8"
}
}
]
DEBUG [job tool] initial work dir {}
INFO [job tool] /tmp/89n9yj5o$ ls \
/tmp/qu6q87wp/stg0fec546f-112c-4c2c-afa0-17256f9bd17a/file \
/tmp/qu6q87wp/stgdcb1db8c-5f78-4354-b36e-915ec91ebab8/directory
ls: cannot access '/tmp/qu6q87wp/stg0fec546f-112c-4c2c-afa0-17256f9bd17a/file': No such file or directory
/tmp/qu6q87wp/stgdcb1db8c-5f78-4354-b36e-915ec91ebab8/directory:
file
DEBUG Could not collect memory usage, job ended before monitoring began.
WARNING [job tool] exited with status: 2
WARNING [job tool] completed permanentFail
DEBUG [job tool] outputs {}
DEBUG [job tool] Removing input staging directory /tmp/qu6q87wp
DEBUG [job tool] Removing temporary directory /tmp/pk_1qhky
DEBUG Removing intermediate output directory /tmp/89n9yj5o
{}WARNING Final process status is permanentFail
```
If we use CWL v1.0, this runs successfully:
```
$ cwltool --debug tool.cwl inputs.yaml
INFO /home/user/Documents/checkouts/cwltool/venv/bin/cwltool 3.1.20221115160822
INFO Resolved 'tool.cwl' to 'file:///home/user/Documents/checkouts/cwltool/my/issue2/tool.cwl'
DEBUG [job tool] initializing from file:///home/user/Documents/checkouts/cwltool/my/issue2/tool.cwl#tool
DEBUG [job tool] {
"directory": {
"class": "Directory",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory",
"basename": "directory",
"listing": [
{
"class": "File",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory/file",
"basename": "file",
"size": 0
}
]
},
"file": {
"class": "File",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory/file",
"size": 0,
"basename": "file",
"nameroot": "file",
"nameext": ""
}
}
DEBUG [job tool] path mappings is {
"file:///home/user/Documents/checkouts/cwltool/my/issue2/directory": [
"/home/user/Documents/checkouts/cwltool/my/issue2/directory",
"/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory",
"Directory",
true
],
"file:///home/user/Documents/checkouts/cwltool/my/issue2/directory/file": [
"/home/user/Documents/checkouts/cwltool/my/issue2/directory/file",
"/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory/file",
"File",
false
]
}
DEBUG [job tool] command line bindings is [
{
"position": [
-1000000,
0
],
"datum": "ls"
},
{
"position": [
1,
"file"
],
"datum": {
"class": "File",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory/file",
"size": 0,
"basename": "file",
"nameroot": "file",
"nameext": "",
"path": "/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory/file",
"dirname": "/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory"
}
},
{
"position": [
2,
"directory"
],
"datum": {
"class": "Directory",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory",
"basename": "directory",
"listing": [
{
"class": "File",
"location": "file:///home/user/Documents/checkouts/cwltool/my/issue2/directory/file",
"basename": "file",
"size": 0,
"path": "/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory/file",
"dirname": "/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory",
"nameroot": "file",
"nameext": ""
}
],
"path": "/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory",
"dirname": "/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208"
}
}
]
DEBUG [job tool] initial work dir {}
INFO [job tool] /tmp/0aycvk8u$ ls \
/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory/file \
/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory
/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory/file
/tmp/4k4zjduc/stg81d0fed8-0aa8-498c-b9c2-54a06ad10208/directory:
file
DEBUG Could not collect memory usage, job ended before monitoring began.
INFO [job tool] completed success
DEBUG [job tool] outputs {}
DEBUG [job tool] Removing input staging directory /tmp/4k4zjduc
DEBUG [job tool] Removing temporary directory /tmp/mt4vd3cg
DEBUG Removing intermediate output directory /tmp/0aycvk8u
{}INFO Final process status is success
```
## Your Environment
* cwltool version:
```
$ cwltool --version
/home/user/Documents/checkouts/cwltool/venv/bin/cwltool 3.1.20221115160822
```
(using the tip of `main`)
Contributor guide
Assessment
This issue has not been assessed yet.