nextflow-io / nextflow-io/nextflow

Unable to override container entrypoint in kubernetes with fusion and wave enabled

Open
#5,764 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

executor/k8s stale storage/fusion
Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

Bug report

We are aware that Nextflow expects images to have /bin/bash ENTRYPOINT by definition. But we are able to override our images entrypoint to fit /bin/bash in local executor environment. Can't achieve this in k8s executor.

Expected behavior and actual behavior

Container entrypoint should be configurable when using Nextflow with k8s, Fusion, and Wave configurations.
Currently, all attempted methods to override container entrypoint fail in this environment combination.

Steps to reproduce the problem
  1. Create a simple Nextflow setup with Fusion and Wave:
wave {
    enabled = true
    endpoint = '${WAVE_URL}'
}

fusion {
    enabled = true
}

workDir = '${S3_WORKDIR}'

k8s {
    namespace = 'nextflow'
    serviceAccount = 'nextflow-sa'
}

process test_process {
    container '${CONTAINER}'
    script:
    """
    echo "hello"
    """
}

workflow {
    test_process()
}
  1. Attempted override methods (all failed):
# Method 1: Process containerOptions
process {
    containerOptions '--entrypoint=""'
}

# Method 2: Configuration settings
k8s {
    entrypointOverride = true
}

docker {
    entrypointOverride = true
}

process {
    entrypointOverride = true
}

# Method 3: Environment variable
export NXF_CONTAINER_ENTRYPOINT_OVERRIDE=true

Reference: Entrypoint override implementation in source code: https://github.com/nextflow-io/nextflow/blob/62263a35c5c3b62d2f9c3d3a8f3a5a9117c41290/modules/nextflow/src/main/groovy/nextflow/k8s/K8sConfig.groovy#L140

Program output

No error messages are shown - the entrypoint override simply does not take effect when using k8s, Fusion, and Wave together.

Environment
  • Nextflow version: 24.10.4 build 5934
  • Java version: openjdk 21-internal 2023-09-19
  • Operating system: Ubuntu 24.04
  • Bash version: 5.1.16(1)-release (x86_64-pc-linux-gnu)
Additional context
  • The entrypoint override works correctly with local executor using containerOptions in the process definition
  • We need this functionality to run containers that have entrypoints different from /bin/bash
  • There is no clear documentation for entrypoint override configuration in k8s

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading modules/nextflow/src/main/groovy/nextflow/k8s/K8sConfig.groovy around line 140 and trace how entrypoint settings are handled with Fusion and Wave enabled. Reproduce the issue with the supplied Nextflow configuration and verify that a supported override changes the Kubernetes container entrypoint in this combination.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, groovy, kubernetes
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.