common-workflow-language / common-workflow-language/cwltool

CWLProv: needs to add running the conformance test to regular testing

Open
#916 1 comment 0 reactions 2 assignees Claimed by @FarahZKhan View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

> 195 tests passed, 2 failures, 0 unsupported features

- [ ] add `./run_test.sh RUNNER=cwltool EXTRA="--provenance $PWD/RO_test"` to ci.commonwl.org
or maybe `seq 1 133 | parallel ./run_test.sh RUNNER=cwltool EXTRA=\"--provenance $PWD/ro_test/RO_{#}\" -n{#}` ?
Bonus: Check packed workflows for any `file:///` references that slipped through
```
find ro_test/ -name packed.cwl | xargs grep "file://$PWD"
find ro_test/ -name packed.cwl | xargs grep path\":
```

---
- [x] 87 "Test directories in secondaryFiles"
- [x] 136 "Test specifying secondaryFiles in subdirectories of the job input document."
- [x] 137 "Test specifying secondaryFiles in same subdirectory of the job input as the primary input file."
```
Traceback (most recent call last):
File "/home/mcrusoe/cwltool/cwltool/executors.py", line 167, in run_jobs
job.run(runtime_context)
File "/home/mcrusoe/cwltool/cwltool/job.py", line 498, in run
self._execute(runtime, env, runtimeContext)
File "/home/mcrusoe/cwltool/cwltool/job.py", line 332, in _execute
outputs, runtimeContext.process_run_id, str(self.name))
File "/home/mcrusoe/cwltool/cwltool/provenance.py", line 839, in generate_output_prov
entity = self.declare_artefact(value)
File "/home/mcrusoe/cwltool/cwltool/provenance.py", line 736, in declare_artefact
(entity, _, _) = self.declare_file(value)
File "/home/mcrusoe/cwltool/cwltool/provenance.py", line 571, in declare_file
(sec_entity, _, _) = self.declare_file(sec)
File "/home/mcrusoe/cwltool/cwltool/provenance.py", line 514, in declare_file
raise ValueError("Must have class:File: %s" % value)
ValueError: Must have class:File: {'basename': 'input.txt_idx8', 'location': 'file:///tmp/tmpvswaqxeb/input.txt_idx8', 'class': 'Directory'}
Workflow error, try again with --debug for more information:
Must have class:File: {'basename': 'input.txt_idx8', 'location': 'file:///tmp/tmpvswaqxeb/input.txt_idx8', 'class': 'Directory'}
```

---

- [x] 61 "Test parameter evaluation, no support for JS expressions"
- [x] 62 "Test parameter evaluation, with support for JS expressions"
- [x] 73 "Test record type output binding."
- [x] 143 "Test record type inputs to and outputs from workflows"
- [x] 160 "Test Any parameter with record input to a tool"
- [x] 165 "Test Any parameter with record input to a tool" (in a workflow)
(`"@id": "urn:uuid:8238136c-4736-44a2-a6cc-16fbc789fdd8"` in the output records

---

- [x] 90 "Test file literal as input"
- [x] 120 "Test file literal as input without Docker"
- [x] 190 "Pipe to stdin from literal File via a Directory literal"
- [x] 191 "Test non-stdin reference to literal File via a Directory literal"
```
/home/mcrusoe/cwltool/env3/bin/cwltool 1.0
Resolved 'v1.0/cat3-tool.cwl' to 'file:///home/mcrusoe/common-workflow-language/v1.0/v1.0/cat3-tool.cwl'
Got workflow error
Traceback (most recent call last):
File "/home/mcrusoe/cwltool/cwltool/executors.py", line 164, in run_jobs
runtime_context.research_obj)
File "/home/mcrusoe/cwltool/cwltool/provenance.py", line 459, in evaluate
self.used_artefacts(customised_job, self.workflow_run_uri)
File "/home/mcrusoe/cwltool/cwltool/provenance.py", line 821, in used_artefacts
entity = self.declare_artefact(value)
File "/home/mcrusoe/cwltool/cwltool/provenance.py", line 736, in declare_artefact
(entity, _, _) = self.declare_file(value)
File "/home/mcrusoe/cwltool/cwltool/provenance.py", line 531, in declare_file
with fsaccess.open(location, "rb") as fhandle:
File "/home/mcrusoe/cwltool/cwltool/stdfsaccess.py", line 41, in open
return open(self._abs(fn), mode)
FileNotFoundError: [Errno 2] No such file or directory: '_:221ad84f-daa1-48b7-bfee-8f28dae22ed8'
Workflow error, try again with --debug for more information:
[Errno 2] No such file or directory: '_:221ad84f-daa1-48b7-bfee-8f28dae22ed8'
```

- [x] 118 "Test empty writable dir with InitialWorkDirRequirement inside Docker"
- [x] 55 "Test InitialWorkDirRequirement linking input files and capturing secondaryFiles on input and output. Also tests the use of a variety of parameter references and expressions in the secondaryFiles field. "
`size: 0` missing, maybe a serialization error?

From CWL v1.1 conformance tests

- [ ] 234 "symlink to file inside of working directory should be retrieved"
```
Traceback (most recent call last):
File "/home/michael/cwltool/cwltool/executors.py", line 188, in run_jobs
job.run(runtime_context)
File "/home/michael/cwltool/cwltool/job.py", line 693, in run
self._execute(runtime, env, runtimeContext, monitor_function)
File "/home/michael/cwltool/cwltool/job.py", line 366, in _execute
outputs, datetime.datetime.now())
File "/home/michael/cwltool/cwltool/provenance.py", line 487, in record_process_end
self.generate_output_prov(outputs, process_run_id, process_name)
File "/home/michael/cwltool/cwltool/provenance.py", line 826, in generate_output_prov
entity = self.declare_artefact(value)
File "/home/michael/cwltool/cwltool/provenance.py", line 715, in declare_artefact
(entity, _, _) = self.declare_file(value)
File "/home/michael/cwltool/cwltool/provenance.py", line 509, in declare_file
relative_path = self.research_object.add_data_file(fhandle)
File "/home/michael/cwltool/cwltool/provenance.py", line 1425, in add_data_file
self._add_to_bagit(rel_path, sha1=checksum)
File "/home/michael/cwltool/cwltool/provenance.py", line 1486, in _add_to_bagit
raise IOError("File %s does not exist within RO: %s" % (rel_path, local_path))
OSError: File data/cd/cd28ec34f3f9425aca544b6332453708e8aaa82a does not exist within RO: /tmp/opj488vu/data/cd/cd28ec34f3f9425aca544b6332453708e8aaa82a
```

- [ ] 236 "inplace update has side effect on directory content"
```
Compare failure expected: {
"a": [
{
"basename": "blurb",
"class": "File",
"location": "blurb"
}
],
"b": [
{
"basename": "blurb",
"class": "File",
"location": "blurb"
}
]
}
got: {
"a": [],
"b": [
{
"basename": "blurb",
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
"class": "File",
"location": "file:///tmp/tmpo9kr4uf3/blurb",
"path": "/tmp/tmpo9kr4uf3/blurb",
"size": 0
}
]
}
```

- [ ] 242 "Test that default behavior is 'no_listing' if not specified"
- [ ] 243 "Test that 'listing' is not present when LoadListingRequirement is 'no_listing'"
- [ ] 244 "Test that 'listing' is not present when loadListing on input parameter is 'no_listing'"
- [ ] 245 "Test that 'listing' is present in top directory object but not subdirectory object when LoadListingRequirement is 'shallow_listing'"
- [ ] 246 "Test that 'listing' is present in top directory object but not subdirectory object when loadListing on input parameter loadListing is 'shallow_listing'"
```
Compare failure expected: {
"out": true
}
got: {
"out": false
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.