nextflow-io / nextflow-io/nextflow
Basic architecture awareness when using singularity/apptainer - pulling
Nobody has claimed this yet.
- Dominant language
- Groovy
- Stars
- 3.5k
- Forks
- 811
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 61
Description
Support features allowing mixed-architecture executions
To facilitate situations where clusters have mixed basic architectures (e.g. GPU nodes are ARM-based while x86-64 is used for the rest of the cluster), it would be great if nextflow could pass through process.arch (if any) to singularity/apptainer pull.
Technically, what I wish for is that
- if the process has an
archset (possibly one that fits a whitelist) asARCHVALUE, that is provided to thepullcommand as--arch ARCHVALUE
doing so would allow getting the right container image from multi-architecture ORAS sources.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing Nextflow's Singularity/Apptainer container pull path and how process.arch is exposed to processes. Verify the requested behavior for an ARCHVALUE, including any supported whitelist, and confirm that the pull command receives --arch ARCHVALUE so multi-architecture ORAS images can be selected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- groovy
- Domain
- hpc
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100