nextflow-io / nextflow-io/nextflow

Nextflow won't work when running from private repo and main script is not named "main.nf"

Open
#4,465 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug lang/config software/git triage/investigate
Dominant language
Groovy
Stars
3.5k
Forks
811
Avg merge
2d 11h
Merged PRs (30d)
61

Description

Bug report

Expected behavior and actual behavior

When running nextflow on a private repo with a main script called other than main.nf, it fails, complaining that the resource main.nf was not found.

If manifest.mainScript = "my_script.nf" is added to the nextflow.config file, it works.

# nextflow.config
params {
    skip_step = false
}
manifest {
    mainScript = "my_script.nf"
}

But if this same line is added by including another config file into the main one, it won't work.

# nextflow.config
params {
    skip_step = false
}
includeConfig 'manifest.config'

# manifest.config
manifest {
    mainScript = "my_script.nf"
}
Steps to reproduce the problem
  1. Set up a git provider (e.g. GitHub) and save it to the $HOME/.nextflow/scm file.
  2. Create a minimal nextflow.config file (optional).
  3. Create a minimal script named my_script.nf.
  4. Save it to a new private repo on your git provider (e.g. GitHub)
  5. Run the following command:
nextflow run https://github.com/my_user/my_repo.git -main-script my_script.nf
Program output
N E X T F L O W  ~  version 23.10.0
Pulling sralchemab/test_repo ...
Remote resource not found: https://api.github.com/repos/sralchemab/test_repo/contents/main.nf

Here's the nextflow.log file.

Additional context

If I run it locally, it works as expected.

Environment
  • Nextflow version: 23.10.0
  • Java version: openjdk 17.0.9-internal 2023-10-17
  • Operating system: Amazon Linux 2023
  • Bash version: zsh 5.8.1 (x86_64-amazon-linux-gnu)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the private-repository command with nextflow.config, manifest.config, and -main-script my_script.nf, then inspect the private GitHub retrieval and included-configuration paths. Done means the command locates my_script.nf when manifest.mainScript is supplied through an included config, without requesting main.nf.

Written by the indexing model from the issue text.

Assessment

Tech stack
github, groovy
Domain
backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.