nextflow-io / nextflow-io/nextflow

Make Nextflow Clean Options Available for Use in Nextflow.config

Open
#5,679 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

New feature

It would be great if the options available to the nextflow clean command (documented here) could be used with cleanup = true in the nextflow.config file.

Usage scenario

I would like to automatically clean up the Nextflow work folders when runs finish successfully while keeping all (typically small) Nextflow logs and metadata files. The nextflow clean command supports this with the -keep-logs option but it seems like there's no way to specify this option in the nextflow.config file so that I don't have to run a separate nextflow command outside of the pipeline.

Suggest implementation

Perhaps the unscoped cleanup configuration option could be turned into a scoped option called clean that includes the options available to the nextflow clean command.

P.S. For people using Fusion, it would also be super useful if these options could work on objects in S3 storage, but I will create a separate request for that.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the existing nextflow clean command and the unscoped cleanup option in nextflow.config, then compare the documented clean options with the available configuration settings. Define how a scoped clean configuration should represent options such as -keep-logs, and verify that successful pipeline runs clean work folders while retaining logs and metadata. Fusion/S3 support is explicitly separate from this request.

Written by the indexing model from the issue text.

Assessment

Tech stack
groovy
Domain
cli, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.