nextflow-io / nextflow-io/nextflow
how to load a Nextflow Plugin from a private git repo
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
in the Nextflow Plugin docs ( https://www.nextflow.io/docs/latest/plugins.html#using-plugins ) it describes config syntax like this to specify a version of a plugin
plugins {
id 'nf-hello@0.1.0'
}
Further down, the "Testing" section describes how to use a custom registry for plugins ( https://www.nextflow.io/docs/latest/plugins.html#testing-plugins )
# custom registry at https://github.com/my-org/plugins
export NXF_PLUGINS_TEST_REPOSITORY="https://raw.githubusercontent.com/my-org/plugins/main/plugins.json"
# custom plugin release
export NXF_PLUGINS_TEST_REPOSITORY="https://github.com/nextflow-io/nf-hello/releases/download/0.3.0/nf-hello-0.3.0-meta.json"
nextflow run <pipeline> -plugins nf-hello
It seems that the "registry" here contains a list of all available Nextflow Plugins; https://github.com/nextflow-io/plugins/blob/main/plugins.json
Its not clear to me how we can combine these methods, such that we can still use the default Nextflow Plugins such as nf-prov from the official sources, and also include some custom modules that may be hosted internally on a private git repository.
Based on these docs, it sounds as if you would have to mirror the entire official registry JSON yourself, and add in the URL to your custom plugin. Its not clear how credentials for the custom plugin would be passed.
I think it would be a lot easier if something like the syntax from pip could be used, it looks like this;
(from conda environment.yml)
- pip:
- git+https://github.com/myorg/my-plugin-repo.git@1.2.3
something like this which lets you bundle both a git repo URL and a version label along with the plugin id might be helpful, is this possible? Are there other methods available to allow for providing the path directly to the plugin to use, such as maybe a S3 path to a .jar file in a bucket somewhere?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Nextflow plugin documentation sections “Using plugins” and “Testing plugins”, then inspect the official plugins.json registry and the NXF_PLUGINS_TEST_REPOSITORY examples mentioned here. Clarify whether official and private plugins can be combined, how credentials are supplied, and whether direct Git, S3, or JAR paths are supported; update the documentation to state the supported approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100