galaxyproject / galaxyproject/planemo

CWL steps that return arrays of files should be discovered datasets

Open
#1,484 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
110
Forks
102
Avg merge
4d 21h
Merged PRs (30d)
13

Description

We would like to test CWL CommandLineTools that produce an output containing an **array of files** discovered during execution. This ought to be the CWL equivalent of [individual datasets](https://planemo.readthedocs.io/en/latest/writing_advanced.html#individual-datasets) in the Advanced Tool Development Topic on dynamic numbers of outputs, where CWL's internal logic provides the same functionality as the Galaxy `discover_datasets` element.

All of the documentation and examples of how to make assertions for CWL outputs seem to only treat the case where the output is a **single file**.

Even though the CommandLineTool doesn't known how many outputs will be made, concretely for every test case we do know what outputs to expect, and can name them explicitly in the assertions.

If we try to use `element_tests` on a set of expected outputs, Planemo raises a TypeError in `verify_elements`, that suggests that Planemo isn't converting the array of files into a data collection as `galaxy/tool_util` expects.

```yaml
- doc: generate some subsets by sampling
job: sample_job.yaml
outputs:
samples:
element_tests:
subset-1.txt:
asserts: {"has_n_lines": {"n": 100}}
subset-2.txt:
asserts: {"has_n_lines": {"n": 100}}
```

The error is
```
File "lib/python3.13/site-packages/galaxy/tool_util/verify/interactor.py", line 1205, in verify_collection
verify_elements(data_collection["elements"], output_collection_def.element_tests)
```

Running Planemo under Pdb reveals that `data_collection` is an array of CWL objects of class `File`, not a data collection that [`verify_collection`](https://github.com/galaxyproject/galaxy/blob/1a91049029b93fa6a20c564a9bb16df5fba7aa7d/lib/galaxy/tool_util/verify/interactor.py#L1205) can consume.

So, in decreasing order, the hope is that
1. Planemo can in fact make assertions about CWL arrays, but we couldn't find it in the documentation. We would be willing to make a PR to improve the documentation.
2. There is a way _in the Planemo test_ to declare that the array of files is a data collection, or coerce it.
3. Planemo needs to be be modified to convert CWL arrays to collections, on which assertions can be expressed. I would need advice about where in the code this should happen, before I could say whether we could help.
4. There is a workaround, that involves using another representation for the array of files. This could be considered but would be costly, since our CWL CommandLineTools really do return arrays that subsequent steps scatter over. Normally I would be reticent to change the representation and the pipelines just to satisfy the testing framework.

Thanks in advance for any advice you might have

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.