common-workflow-language / common-workflow-language/cwl-v1.2

Implement clarifications and new conformance tests for 1.2.1

Open
#54 3 comments 1 reaction 0 assignees View on GitHub
help wanted
Dominant language
Common Workflow Language
Stars
45
Forks
28
PR merge metrics
No merged PRs in 30d

Description

requests to fix any of the below should go against https://github.com/common-workflow-language/cwl-v1.2/tree/1.2.1_proposed You can [preview all the implemented changes for v1.2.1 so far](https://github.com/common-workflow-language/cwl-v1.2/pull/115) but know that they have not been ratified by the CWL Leadership Team; additional comments are requested.

Anyone is welcome to take one of these and start adding the needed conformance tests and specification clarifications! Please just drop a note in the linked issue when you start.

If you have never written a CWL conformance test or edited the text of the CWL standards before, that's okay! Just ask for instructions on what to do in the linked issue of your choice.

## Candidates issues
Rough consensus with a known solution

### New conformance tests

See https://github.com/common-workflow-language/cwl-v1.2/blob/main/CONFORMANCE_TESTS.md#format-of-the-conformance-test-file for information on how to write a new conformance test

ToDo

- [ ] [Conformance tests for renaming output files with basename](https://github.com/common-workflow-language/common-workflow-language/issues/900)
- [ ] confirm that fixing the above also covers everything in [Clarify the description of `basename` modification in the spec](https://github.com/common-workflow-language/common-workflow-language/issues/881)
- [ ] [Add tests for itemSeparator](https://github.com/common-workflow-language/common-workflow-language/issues/834)
- [ ] [Conformance test for optional record input](https://github.com/common-workflow-language/common-workflow-language/issues/886)
- [ ] [need conformance test for workflow output + linkMerge: merge_flattened](https://github.com/common-workflow-language/common-workflow-language/issues/795)
- [ ] [There are no specific conformance tests about the behavior for nested inputBinding](https://github.com/common-workflow-language/common-workflow-language/issues/783)
- [ ] [Add conformance test for ECMAScript 5.1 strict mode](https://github.com/common-workflow-language/common-workflow-language/issues/696)
- [ ] [create conformance test for InitialWorkDirRequirement w/ & w/o Docker and reference to input file paths](https://github.com/common-workflow-language/common-workflow-language/issues/692)
- [ ] Continue with [Audit the objects available for every place we allow parameter references/expressions](https://github.com/common-workflow-language/common-workflow-language/issues/705) producing new conformance tests for each category
- [ ] [new conformance test: symlinks using absolute paths in a Directory](https://github.com/common-workflow-language/cwltool/issues/1461#issue-929273041)
- [ ] [two directory inputs with same basename but different paths](https://dev.arvados.org/issues/18723)
- [ ] `dockerLoad`, `dockerFile`, and `dockerImport` lack conformance tests
- [ ] workflow in subdirectory, tools in another subdirectory, `run` contains relative paths like `run: ../tools/my_tool.cwl`. [Example](https://github.com/athenarc/whole-exome-seq/blob/main/workflows/Whole_Exome-Seq.cwl); source: https://github.com/DataBiosphere/toil/pull/4052#issuecomment-1149705565
- [ ] [test for parameter references in CommandLineTool.arguments](https://github.com/common-workflow-language/common-workflow-language/pull/706) (**pull request**)
- ~~[a proper test for WorkReuse](https://github.com/common-workflow-language/cwl-v1.2/issues/86)~~ (**not possible with cwltest**)
- [ ] [potential secondaryFiles conformance test candidate](https://github.com/common-workflow-language/common-workflow-language/issues/702) (**cannot reproduce with cwltool**)

### Specification clarification
changes to wording of the specification, may require new conformance test(s) to clarify or confirm

ToDo

- [ ] [Specification is unclear on whether array typeDefs are allowed in SchemaDefRequirement](https://github.com/common-workflow-language/cwl-v1.2/issues/53)
- [ ] [Document that Directory loadListing behavior in v1.1 is not backwards compatible](https://github.com/common-workflow-language/common-workflow-language/issues/893)
- [ ] [Clarify records with multiple inputBindings](https://github.com/common-workflow-language/common-workflow-language/issues/890)
- [ ] [Several terms are used for the value of record type: objects, maps, and records](https://github.com/common-workflow-language/common-workflow-language/issues/775)
- [ ] [Define decoding method for loadContents and cwl.output.json](https://github.com/common-workflow-language/common-workflow-language/issues/736)
- [ ] [clarify use of parameter references / expressions in ResourceRequirement fields](https://github.com/common-workflow-language/common-workflow-language/issues/707)
- [ ] [Does CommandLineBinding.loadContents apply to secondary files?](https://github.com/common-workflow-language/common-workflow-language/issues/644)
- [ ] [Expected behavior of outputBinding on array of file](https://github.com/common-workflow-language/common-workflow-language/issues/643)
- [ ] [be explicit about how to format an input object file via the optional cwl-runner CLI](https://github.com/common-workflow-language/common-workflow-language/issues/525)
- [ ] [Improve explanations about array level and per-item inputBinding](https://github.com/common-workflow-language/common-workflow-language/issues/411)
- [ ] [better document the 'default' and 'valueFrom' fields on WorkflowStepInput](https://github.com/common-workflow-language/common-workflow-language/issues/359)
- [ ] [clarify type checking rules](https://github.com/common-workflow-language/common-workflow-language/issues/205)
- [ ] [WorkflowOutputParameter outputSource should not be optional](https://github.com/common-workflow-language/cwl-v1.2/issues/98)
- [ ] [Examples for linkMerge merge_flattened](https://github.com/common-workflow-language/cwl-v1.2/issues/96)
- [ ] [specify YAML version, which schema](https://github.com/common-workflow-language/cwl-v1.2/issues/93)
- [ ] [fields not specified in an input object get a value of 'null'](https://github.com/common-workflow-language/cwl-v1.2/issues/87)
- [ ] [clarify sort order for `glob`s](https://github.com/common-workflow-language/cwl-v1.2/issues/233)
- [ ] [clarify LoadListingEnum](https://github.com/common-workflow-language/cwl-v1.2/issues/75#issuecomment-741656667)
- [ ] [[prose] Confirm that final output objects must have the listings of all directories recursively expanded](https://github.com/common-workflow-language/cwl-v1.2/issues/108)
- [ ] [explain how to render a `type: enum` on the command line](https://github.com/common-workflow-language/common-workflow-language/issues/936)
- [ ] [clarify evaluation policy of Workflow level requirements](https://github.com/common-workflow-language/cwl-v1.2/issues/157)
- [ ] [Explain](https://github.com/common-workflow-language/cwl-website/issues/136) the concept of a _Abstract CWL Workflow_, a `class: Workflow` with one or more `class: Operation` steps (or sub-steps). An _Abstract CWL Workflow_ description is not (fully) executable, as `Operation` processes are themselves abstract; but it does allow describing a workflow and providing a skeleton to attach metadata about the steps, authorship, tools used, and more. _Abstract CWL Workflows_ are used in the RO-Crate [workflows](https://w3id.org/workflowhub/workflow-ro-crate/) and [workflow runs](https://www.researchobject.org/workflow-run-crate/) profiles; the former used by workflowhub.eu.

## Bonus issues
These are worthy, but we don't have a solution ready or obvious plan yet

Pending Discussion:

- [ ] [directory with executable script inside](https://github.com/common-workflow-language/cwl-v1.2/issues/90) → https://github.com/common-workflow-language/cwl-v1.2/pull/263

ToDo

- [ ] [JS expression returning Undefined](https://github.com/common-workflow-language/cwl-v1.2/issues/97)
- [ ] [document format schema bridging technique](https://github.com/common-workflow-language/common-workflow-language/issues/765)
- [ ] [Definition of `hints` is inconsistent](https://github.com/common-workflow-language/common-workflow-language/issues/896)
- [ ] [Clarify, make explicit, and add conformance test if CommandLineTool requires a baseCommand vs being a no-op](https://github.com/common-workflow-language/common-workflow-language/issues/818)
- [ ] [default values with secondaryfiles defined by extension needs a conformance test](https://github.com/common-workflow-language/common-workflow-language/issues/670)
- [ ] ["Edit This Section" button for the rendered CWL standards](https://github.com/common-workflow-language/common-workflow-language/issues/917)

### Done
- [x] [outputEval: "time passed" syntax error](https://github.com/common-workflow-language/cwl-v1.2/issues/51)
- [x] [Allow `[]` `?` in types under ArraySchema.list](https://github.com/common-workflow-language/cwltool/pull/1515)
- [x] [conformance tests: add `docker.io/` to `dockerPull` entries that lack an explicit registry](https://github.com/common-workflow-language/cwl-v1.2/issues/119)
- [x] [clarify $(..) vs ${...}](https://github.com/common-workflow-language/common-workflow-language/issues/206)
- [x] [conformance test requires `location: Any` what does it mean ?](https://github.com/common-workflow-language/common-workflow-language/issues/740) (in progress at https://github.com/common-workflow-language/cwl-v1.2/pull/124)
- [x] [testing InitialWorkDirRequirement with directories as output and more](https://github.com/common-workflow-language/common-workflow-language/issues/749) (in progress at https://github.com/common-workflow-language/cwl-v1.2/pull/126 )
- [x] [Conflict between the conformance test 61 and the spec for parameter references](https://github.com/common-workflow-language/common-workflow-language/issues/761) (in progress at https://github.com/common-workflow-language/cwl-v1.2/pull/64)
- [x] [new conformance test: $(runtime.outdir) for outputBinding glob](https://github.com/common-workflow-language/cwl-v1.2/issues/109) https://github.com/common-workflow-language/cwl-v1.2/pull/164/
- [x] [Unclear specification of `run` field in the workflow description](https://github.com/common-workflow-language/common-workflow-language/issues/760)
- [x] [mention `cwl-upgrader`; explain version mixing](https://github.com/common-workflow-language/cwl-v1.2/issues/101) https://github.com/common-workflow-language/cwl-v1.2/pull/172
- [x] [fix the broken InitialWorkDirRequirement link](https://github.com/common-workflow-language/cwl-v1.1/pull/71/files)
- [x] remove the `initial_workdir_output_glob_duplicate` test common-workflow-language/common-workflow-language#160
- [x] The ID numbers are off by one at 220 or before Irrelvant due to common-workflow-language/cwltest#110
- [x] [conformance test for input record + JS](https://github.com/common-workflow-language/common-workflow-language/issues/755) https://github.com/common-workflow-language/cwl-v1.2/pull/170
- [x] [New SchemaDefRequirement conformance test candidate](https://github.com/common-workflow-language/common-workflow-language/issues/703) https://github.com/common-workflow-language/cwl-v1.2/pull/171
- [x] [conformance test needed: MultipleInputFeatureRequirement on workflow outputs](https://github.com/common-workflow-language/common-workflow-language/issues/637) https://github.com/common-workflow-language/cwl-v1.2/pull/169
- [x] [Define the concept of an "opaque string"](https://github.com/common-workflow-language/common-workflow-language/issues/914) https://github.com/common-workflow-language/cwl-v1.2/pull/162
- [x] [affirm that WorkflowOutputParameter.outputSource can reference workflow level inputs](https://github.com/common-workflow-language/common-workflow-language/issues/763) https://github.com/common-workflow-language/cwl-v1.2/pull/178
- [x] [Stdout behavior with chained commands](https://github.com/common-workflow-language/cwl-v1.2/issues/95) https://github.com/common-workflow-language/cwl-v1.2/pull/190
- [x] [clarify meaning of Process.id](https://github.com/common-workflow-language/cwl-v1.2/issues/117) https://github.com/common-workflow-language/cwl-v1.2/pull/166
- [x] [Why conformance test 63 expect empty result ?](https://github.com/common-workflow-language/common-workflow-language/issues/743) https://github.com/common-workflow-language/cwl-v1.2/pull/184
- [x] https://github.com/common-workflow-language/cwl-v1.2/issues/193
- [x] #239
- [x] [Prose for `location` in File/Directory mentions URIs instead of IRIs](https://github.com/common-workflow-language/cwl-v1.2/issues/100)
- [x] [About specification when output class is File](https://github.com/common-workflow-language/common-workflow-language/issues/629)
- [x] common-workflow-language/common-workflow-language#232

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with CONFORMANCE_TESTS.md, especially the format of the conformance test file, then choose one candidate issue and read the relevant discussion and the 1.2.1_proposed branch. Done means the selected specification clarification and/or conformance test is completed and the linked issue is updated.

Written by the indexing model from the issue text.

Assessment

Domain
documentation, testing
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.