galaxyproject / galaxyproject/planemo

Workflow testing reports number of tests incorrectly

Open
#886 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
110
Forks
102
Avg merge
4d 21h
Merged PRs (30d)
13

Description

The following test.yml for the training material quality-control tutorial workflow provides a good illustration of the problem:

```
---
- doc: "Test sample data for Sequence-analyses:Quality-control"
job:
reads_1:
class: File
location: https://zenodo.org/record/61771/files/GSM461178_untreat_paired_subset_1.fastq
filetype: fastqsanger
reads_2:
class: File
location: https://zenodo.org/record/61771/files/GSM461178_untreat_paired_subset_2.fastq
filetype: fastqsanger
outputs:
reads_1_fastqc:
asserts:
has_text:
text: 'Sanger / Illumina 1.9'
reads_1_cutadapt:
asserts:
has_text:
text: '3,651,491 bp (98.7%)'
reads_2_fastqc:
asserts:
has_text:
text: 'Sequence length 37'
reads_cutadapt:
asserts:
has_text:
text: '98,624 (98.6%)'
multiqc:
asserts:
has_text:
text: 'FastQC'
```

This workflow test checks properties of 5 different output files, yet planemo reports:
`All 1 test(s) successfully executed.`

5 test(s) executed is what I would have expected.

Contributor guide

Open the contributing guide

Research direction

Start with the test.yml workflow example in the issue and reproduce the report showing one test for five output files. Trace the workflow-test reporting entry point in planemo, then verify that the completed report counts the asserted output files individually and still reports successful execution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.