nextflow-io / nextflow-io/nextflow

optional parameter for secret directive

Open
#5,243 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

lang/processes secrets
Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

New feature

I want to run an optional process only if the user has provided a secret. Example code:

workflow SUMMARY_WF { 
    main:
    OPTIONAL_PROCESS()
}

process OPTIONAL_PROCESS {
    secret "API_KEY"
  
    output:
    path "output/"

    script:
    """
    run_job.py
    """
}

Currently, if the user has not set the secret for OPTIONAL_PROCESS, the pipeline dies with the following error:

ERROR ~ Error executing process > 'SUMMARY_WF:OPTIONAL_PROCESS (1)'

Caused by:
  Required secret is missing: 'API_KEY'

As far as I can tell from the Nextflow docs, the secret: directive does not include an optional parameter.

Usage scenario

To allow secrets to be optional instead of required. If the secret is not set for the target process, then the environmental variable is empty, and the pipeline developer can thus include a check for the env variable in their code (e.g., if os.getenv("API_KEY") is None:).

Suggest implementation

Add an optional parameter to the secrets: directive.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing how the secret directive handles a missing API_KEY and where the “Required secret is missing” error is produced. Implement the requested optional behavior so an unset secret reaches the process as an empty environment variable, then verify the example process can decide whether to run.

Written by the indexing model from the issue text.

Assessment

Tech stack
groovy
Domain
tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.