common-workflow-language / common-workflow-language/cwl-v1.2
Implement clarifications and new conformance tests for 1.2.1
- Dominant language
- Common Workflow Language
- Stars
- 45
- Forks
- 28
- PR merge metrics
- No merged PRs in 30d
Description
requests to fix any of the below should go against https://github.com/common-workflow-language/cwl-v1.2/tree/1.2.1_proposed You can [preview all the implemented changes for v1.2.1 so far](https://github.com/common-workflow-language/cwl-v1.2/pull/115) but know that they have not been ratified by the CWL Leadership Team; additional comments are requested.
Anyone is welcome to take one of these and start adding the needed conformance tests and specification clarifications! Please just drop a note in the linked issue when you start.
If you have never written a CWL conformance test or edited the text of the CWL standards before, that's okay! Just ask for instructions on what to do in the linked issue of your choice.
## Candidates issues
Rough consensus with a known solution
### New conformance tests
See https://github.com/common-workflow-language/cwl-v1.2/blob/main/CONFORMANCE_TESTS.md#format-of-the-conformance-test-file for information on how to write a new conformance test
ToDo
- [ ] [Conformance tests for renaming output files with basename](https://github.com/common-workflow-language/common-workflow-language/issues/900)
- [ ] confirm that fixing the above also covers everything in [Clarify the description of `basename` modification in the spec](https://github.com/common-workflow-language/common-workflow-language/issues/881)
- [ ] [Add tests for itemSeparator](https://github.com/common-workflow-language/common-workflow-language/issues/834)
- [ ] [Conformance test for optional record input](https://github.com/common-workflow-language/common-workflow-language/issues/886)
- [ ] [need conformance test for workflow output + linkMerge: merge_flattened](https://github.com/common-workflow-language/common-workflow-language/issues/795)
- [ ] [There are no specific conformance tests about the behavior for nested inputBinding](https://github.com/common-workflow-language/common-workflow-language/issues/783)
- [ ] [Add conformance test for ECMAScript 5.1 strict mode](https://github.com/common-workflow-language/common-workflow-language/issues/696)
- [ ] [create conformance test for InitialWorkDirRequirement w/ & w/o Docker and reference to input file paths](https://github.com/common-workflow-language/common-workflow-language/issues/692)
- [ ] Continue with [Audit the objects available for every place we allow parameter references/expressions](https://github.com/common-workflow-language/common-workflow-language/issues/705) producing new conformance tests for each category
- [ ] [new conformance test: symlinks using absolute paths in a Directory](https://github.com/common-workflow-language/cwltool/issues/1461#issue-929273041)
- [ ] [two directory inputs with same basename but different paths](https://dev.arvados.org/issues/18723)
- [ ] `dockerLoad`, `dockerFile`, and `dockerImport` lack conformance tests
- [ ] workflow in subdirectory, tools in another subdirectory, `run` contains relative paths like `run: ../tools/my_tool.cwl`. [Example](https://github.com/athenarc/whole-exome-seq/blob/main/workflows/Whole_Exome-Seq.cwl); source: https://github.com/DataBiosphere/toil/pull/4052#issuecomment-1149705565
- [ ] [test for parameter references in CommandLineTool.arguments](https://github.com/common-workflow-language/common-workflow-language/pull/706) (**pull request**)
- ~~[a proper test for WorkReuse](https://github.com/common-workflow-language/cwl-v1.2/issues/86)~~ (**not possible with cwltest**)
- [ ] [potential secondaryFiles conformance test candidate](https://github.com/common-workflow-language/common-workflow-language/issues/702) (**cannot reproduce with cwltool**)
### Specification clarification
changes to wording of the specification, may require new conformance test(s) to clarify or confirm
ToDo
- [ ] [Specification is unclear on whether array typeDefs are allowed in SchemaDefRequirement](https://github.com/common-workflow-language/cwl-v1.2/issues/53)
- [ ] [Document that Directory loadListing behavior in v1.1 is not backwards compatible](https://github.com/common-workflow-language/common-workflow-language/issues/893)
- [ ] [Clarify records with multiple inputBindings](https://github.com/common-workflow-language/common-workflow-language/issues/890)
- [ ] [Several terms are used for the value of record type: objects, maps, and records](https://github.com/common-workflow-language/common-workflow-language/issues/775)
- [ ] [Define decoding method for loadContents and cwl.output.json](https://github.com/common-workflow-language/common-workflow-language/issues/736)
- [ ] [clarify use of parameter references / expressions in ResourceRequirement fields](https://github.com/common-workflow-language/common-workflow-language/issues/707)
- [ ] [Does CommandLineBinding.loadContents apply to secondary files?](https://github.com/common-workflow-language/common-workflow-language/issues/644)
- [ ] [Expected behavior of outputBinding on array of file](https://github.com/common-workflow-language/common-workflow-language/issues/643)
- [ ] [be explicit about how to format an input object file via the optional cwl-runner CLI](https://github.com/common-workflow-language/common-workflow-language/issues/525)
- [ ] [Improve explanations about array level and per-item inputBinding](https://github.com/common-workflow-language/common-workflow-language/issues/411)
- [ ] [better document the 'default' and 'valueFrom' fields on WorkflowStepInput](https://github.com/common-workflow-language/common-workflow-language/issues/359)
- [ ] [clarify type checking rules](https://github.com/common-workflow-language/common-workflow-language/issues/205)
- [ ] [WorkflowOutputParameter outputSource should not be optional](https://github.com/common-workflow-language/cwl-v1.2/issues/98)
- [ ] [Examples for linkMerge merge_flattened](https://github.com/common-workflow-language/cwl-v1.2/issues/96)
- [ ] [specify YAML version, which schema](https://github.com/common-workflow-language/cwl-v1.2/issues/93)
- [ ] [fields not specified in an input object get a value of 'null'](https://github.com/common-workflow-language/cwl-v1.2/issues/87)
- [ ] [clarify sort order for `glob`s](https://github.com/common-workflow-language/cwl-v1.2/issues/233)
- [ ] [clarify LoadListingEnum](https://github.com/common-workflow-language/cwl-v1.2/issues/75#issuecomment-741656667)
- [ ] [[prose] Confirm that final output objects must have the listings of all directories recursively expanded](https://github.com/common-workflow-language/cwl-v1.2/issues/108)
- [ ] [explain how to render a `type: enum` on the command line](https://github.com/common-workflow-language/common-workflow-language/issues/936)
- [ ] [clarify evaluation policy of Workflow level requirements](https://github.com/common-workflow-language/cwl-v1.2/issues/157)
- [ ] [Explain](https://github.com/common-workflow-language/cwl-website/issues/136) the concept of a _Abstract CWL Workflow_, a `class: Workflow` with one or more `class: Operation` steps (or sub-steps). An _Abstract CWL Workflow_ description is not (fully) executable, as `Operation` processes are themselves abstract; but it does allow describing a workflow and providing a skeleton to attach metadata about the steps, authorship, tools used, and more. _Abstract CWL Workflows_ are used in the RO-Crate [workflows](https://w3id.org/workflowhub/workflow-ro-crate/) and [workflow runs](https://www.researchobject.org/workflow-run-crate/) profiles; the former used by workflowhub.eu.
## Bonus issues
These are worthy, but we don't have a solution ready or obvious plan yet
Pending Discussion:
- [ ] [directory with executable script inside](https://github.com/common-workflow-language/cwl-v1.2/issues/90) → https://github.com/common-workflow-language/cwl-v1.2/pull/263
ToDo
- [ ] [JS expression returning Undefined](https://github.com/common-workflow-language/cwl-v1.2/issues/97)
- [ ] [document format schema bridging technique](https://github.com/common-workflow-language/common-workflow-language/issues/765)
- [ ] [Definition of `hints` is inconsistent](https://github.com/common-workflow-language/common-workflow-language/issues/896)
- [ ] [Clarify, make explicit, and add conformance test if CommandLineTool requires a baseCommand vs being a no-op](https://github.com/common-workflow-language/common-workflow-language/issues/818)
- [ ] [default values with secondaryfiles defined by extension needs a conformance test](https://github.com/common-workflow-language/common-workflow-language/issues/670)
- [ ] ["Edit This Section" button for the rendered CWL standards](https://github.com/common-workflow-language/common-workflow-language/issues/917)
### Done
- [x] [outputEval: "time passed" syntax error](https://github.com/common-workflow-language/cwl-v1.2/issues/51)
- [x] [Allow `[]` `?` in types under ArraySchema.list](https://github.com/common-workflow-language/cwltool/pull/1515)
- [x] [conformance tests: add `docker.io/` to `dockerPull` entries that lack an explicit registry](https://github.com/common-workflow-language/cwl-v1.2/issues/119)
- [x] [clarify $(..) vs ${...}](https://github.com/common-workflow-language/common-workflow-language/issues/206)
- [x] [conformance test requires `location: Any` what does it mean ?](https://github.com/common-workflow-language/common-workflow-language/issues/740) (in progress at https://github.com/common-workflow-language/cwl-v1.2/pull/124)
- [x] [testing InitialWorkDirRequirement with directories as output and more](https://github.com/common-workflow-language/common-workflow-language/issues/749) (in progress at https://github.com/common-workflow-language/cwl-v1.2/pull/126 )
- [x] [Conflict between the conformance test 61 and the spec for parameter references](https://github.com/common-workflow-language/common-workflow-language/issues/761) (in progress at https://github.com/common-workflow-language/cwl-v1.2/pull/64)
- [x] [new conformance test: $(runtime.outdir) for outputBinding glob](https://github.com/common-workflow-language/cwl-v1.2/issues/109) https://github.com/common-workflow-language/cwl-v1.2/pull/164/
- [x] [Unclear specification of `run` field in the workflow description](https://github.com/common-workflow-language/common-workflow-language/issues/760)
- [x] [mention `cwl-upgrader`; explain version mixing](https://github.com/common-workflow-language/cwl-v1.2/issues/101) https://github.com/common-workflow-language/cwl-v1.2/pull/172
- [x] [fix the broken InitialWorkDirRequirement link](https://github.com/common-workflow-language/cwl-v1.1/pull/71/files)
- [x] remove the `initial_workdir_output_glob_duplicate` test common-workflow-language/common-workflow-language#160
- [x] The ID numbers are off by one at 220 or before Irrelvant due to common-workflow-language/cwltest#110
- [x] [conformance test for input record + JS](https://github.com/common-workflow-language/common-workflow-language/issues/755) https://github.com/common-workflow-language/cwl-v1.2/pull/170
- [x] [New SchemaDefRequirement conformance test candidate](https://github.com/common-workflow-language/common-workflow-language/issues/703) https://github.com/common-workflow-language/cwl-v1.2/pull/171
- [x] [conformance test needed: MultipleInputFeatureRequirement on workflow outputs](https://github.com/common-workflow-language/common-workflow-language/issues/637) https://github.com/common-workflow-language/cwl-v1.2/pull/169
- [x] [Define the concept of an "opaque string"](https://github.com/common-workflow-language/common-workflow-language/issues/914) https://github.com/common-workflow-language/cwl-v1.2/pull/162
- [x] [affirm that WorkflowOutputParameter.outputSource can reference workflow level inputs](https://github.com/common-workflow-language/common-workflow-language/issues/763) https://github.com/common-workflow-language/cwl-v1.2/pull/178
- [x] [Stdout behavior with chained commands](https://github.com/common-workflow-language/cwl-v1.2/issues/95) https://github.com/common-workflow-language/cwl-v1.2/pull/190
- [x] [clarify meaning of Process.id](https://github.com/common-workflow-language/cwl-v1.2/issues/117) https://github.com/common-workflow-language/cwl-v1.2/pull/166
- [x] [Why conformance test 63 expect empty result ?](https://github.com/common-workflow-language/common-workflow-language/issues/743) https://github.com/common-workflow-language/cwl-v1.2/pull/184
- [x] https://github.com/common-workflow-language/cwl-v1.2/issues/193
- [x] #239
- [x] [Prose for `location` in File/Directory mentions URIs instead of IRIs](https://github.com/common-workflow-language/cwl-v1.2/issues/100)
- [x] [About specification when output class is File](https://github.com/common-workflow-language/common-workflow-language/issues/629)
- [x] common-workflow-language/common-workflow-language#232
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with CONFORMANCE_TESTS.md, especially the format of the conformance test file, then choose one candidate issue and read the relevant discussion and the 1.2.1_proposed branch. Done means the selected specification clarification and/or conformance test is completed and the linked issue is updated.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation, testing
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100