nf-core / nf-core/tools

Investigate reducing base container size for wave provisioned containers.

Open
#4,315 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

infrastructure
Dominant language
Python
Stars
322
Forks
255
Avg merge
2d 3h
Merged PRs (30d)
5

Description

Currently the base images for wave containers are either pixi or micromamba and they can be relatively bloated when only installing a single small tool.

In pipelines with many processes, which might be run hundreds of times this can cause issues in some cases - e.g. on AWS there can be timeout and EBS storage issues when trying to pull dozens of containers.

An example of this size can be seen in the seqera/wave vs bioconda image sizes:

271.18 MB quay.io/biocontainers/jasminesv
  1.51 GB wave - pixi base
  1.84 GB wave - conda base  
  2.22 GB manual-build-seqera-conda-dockerfile

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin by comparing the current Wave base-image choices, pixi and micromamba, with the cited quay.io/biocontainers/jasminesv image and the AWS pull, timeout, and EBS concerns. Define and validate a smaller base-image approach that still supports single-tool installs and reduces image size; the issue names no files or tests to run.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, infrastructure
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.