Incorrect if-statement logic in nf-core modules test output
Open
Beginner friendly
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 322
- Forks
- 255
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 5
Description
Description of the bug
When running nf-core modules test <tool> I ran into the issue that all tests ran succesfully, but I got an error message:
CRITICAL Ran, but found errors:
- nf-test failed
- nf-test failed
Looking into the code, I found that in the display_nftest_output function of the tests the if-statement seems incorrect. If the "Different Snapshot" is not in the nftest_err message, it will always print the "nf-test failed" message. So when you have no error message (all tests succesfull), it prints that.
Additionally, if you run nf-core modules test <tool> --no-prompts you go into the top part of the if-statement and I correctly see that all tests passed, so it is only a problem if you run without --no-prompts
def display_nftest_output(self, nftest_out: bytes, nftest_err: bytes) -> None:
if self.no_prompts:
# In no_prompts mode (e.g., tests), use simple logging to avoid Rich hanging in non-TTY environments
log.debug("nf-test output:\n%s", nftest_out.decode())
if nftest_err:
log.debug("nf-test error:\n%s", nftest_err.decode())
if "Different Snapshot:" in nftest_err.decode() and self.update:
log.info("Updating snapshot")
self.generate_snapshot()
else:
# Interactive mode: use Rich formatting
print("Displaying nf-test output")
nftest_output = Text.from_ansi(nftest_out.decode())
print(Panel(nftest_output, title="nf-test output"))
print("Displaying nf-test error", nftest_err.decode())
if nftest_err:
print("Parsing nf-test error")
syntax = Syntax(nftest_err.decode(), "diff", theme="ansi_dark")
print("Parsed nf-test error")
print(Panel(syntax, title="nf-test error"))
print("Displaying nf-test error")
if "Different Snapshot:" in nftest_err.decode():
log.error("nf-test failed due to differences in the snapshots")
if self.update is None:
answer = Confirm.ask(
"[bold][blue]?[/] nf-test found differences in the snapshot. Do you want to update it?",
default=True,
)
if answer:
log.info("Updating snapshot")
self.update = True
else:
log.debug("Snapshot not updated")
if self.update:
# update snapshot using nf-test --update-snapshot
self.generate_snapshot()
else:
self.errors.append("nf-test failed")
Command used and terminal output
nf-core modules test nasvar/pipeline --profile docker
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 4.0.3 - https://nf-co.re
INFO Generating nf-test snapshot
Displaying nf-test output
╭────────────────────────────────────────────────────────────────────────────────────────────────────── nf-test output ───────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 🚀 nf-test 0.9.5 │
│ https://www.nf-test.com │
│ Please cite: https://doi.org/10.1093/gigascience/giaf130 │
│ (c) 2021 - 2026 Lukas Forer and Sebastian Schoenherr │
│ │
│ Load .nf-test/plugins/nft-anndata/0.4.1/nft-anndata-0.4.1.jar │
│ Load .nf-test/plugins/nft-bam/0.6.1/nft-bam-0.6.1.jar │
│ Load .nf-test/plugins/nft-csv/0.1.0/nft-csv-0.1.0.jar │
│ Load .nf-test/plugins/nft-compress/0.1.0/nft-compress-0.1.0.jar │
│ Load .nf-test/plugins/nft-fastq/0.1.0/nft-fastq-0.1.0.jar │
│ Load .nf-test/plugins/nft-utils/0.0.9/nft-utils-0.0.9.jar │
│ Load .nf-test/plugins/nft-vcf/1.0.7/nft-vcf-1.0.7.jar │
│ │
│ Test Process NASVAR_PIPELINE │
│ │
│ Test [f427fc01] 'nasvar pipeline - bam - RUNX1::ETV6' PASSED (150.613s) │
│ Test [9a2062a3] 'nasvar pipeline - bam - stub' PASSED (3.621s) │
│ │
│ │
│ SUCCESS: Executed 2 tests in 154.287s │
│ │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Displaying nf-test error
INFO Generating nf-test snapshot again to check stability
Displaying nf-test output
╭────────────────────────────────────────────────────────────────────────────────────────────────────── nf-test output ───────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 🚀 nf-test 0.9.5 │
│ https://www.nf-test.com │
│ Please cite: https://doi.org/10.1093/gigascience/giaf130 │
│ (c) 2021 - 2026 Lukas Forer and Sebastian Schoenherr │
│ │
│ Load .nf-test/plugins/nft-anndata/0.4.1/nft-anndata-0.4.1.jar │
│ Load .nf-test/plugins/nft-bam/0.6.1/nft-bam-0.6.1.jar │
│ Load .nf-test/plugins/nft-csv/0.1.0/nft-csv-0.1.0.jar │
│ Load .nf-test/plugins/nft-compress/0.1.0/nft-compress-0.1.0.jar │
│ Load .nf-test/plugins/nft-fastq/0.1.0/nft-fastq-0.1.0.jar │
│ Load .nf-test/plugins/nft-utils/0.0.9/nft-utils-0.0.9.jar │
│ Load .nf-test/plugins/nft-vcf/1.0.7/nft-vcf-1.0.7.jar │
│ │
│ Test Process NASVAR_PIPELINE │
│ │
│ Test [f427fc01] 'nasvar pipeline - bam - RUNX1::ETV6' PASSED (143.381s) │
│ Test [9a2062a3] 'nasvar pipeline - bam - stub' PASSED (3.259s) │
│ │
│ │
│ SUCCESS: Executed 2 tests in 146.689s │
│ │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Displaying nf-test error
CRITICAL Ran, but found errors:
- nf-test failed
- nf-test failed
(.venv) (base) k.v.cammel@MB-GXG4Q2Q5R6 modules % nf-core modules test nasvar/pipeline --profile docker --no-prompts
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 4.0.3 - https://nf-co.re
INFO Generating nf-test snapshot
INFO Generating nf-test snapshot again to check stability
INFO All tests passed!
System information
Nextflow version 26.04
Nf-test 0.9.5
nf-core 4.0.2 & 4.0.3 (also tried it with the dev version)
Ran on macbook with --profile docker
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in nf_core/components/components_test.py at display_nftest_output, then reproduce the behavior with nf-core modules test without --no-prompts. Check the interactive handling of an empty nf-test error output and make the command report successful tests without adding an error; confirm the shown successful command no longer prints "nf-test failed".
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100