conda-forge / conda-forge/conda-forge.github.io

Testing packages in minimal containers and creating containers on the fly

未关闭
#561 0 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
JavaScript
星标
170
派生
320
平均合并
2 天 10 小时
30 天内合并 PR
5

描述

Hi all.

@jakirkham asks me to write about our work on Containers in https://github.com/conda-forge/staged-recipes/pull/4838#discussion_r180283102

Here we go:

In Bioconda we are using a busybox container to test all our recipes. This is very useful for catching errors like missing zlib or missing libgcc for example and it ensures it runs everywhere. As a nice side-effect we are building optimized and minimal containers. Many containers are just a few MB in size.

A preprint about this project can be found here: https://www.biorxiv.org/content/early/2017/10/11/200683 (biology motivated, but the technical concept is universal)

We have created to nearly 3000 packages containers, with multiple versions.
Please feel free to ask any question and provide feedback.

## Single package containers:

These are easy. We are using a build-container with miniconda installed to install (the previously locally build package from the local channel) into the build-container. We than move everything that is needed to the runtime container - in this case busybox - and we give the container a predictable name. So that everyone can `guess` the name without knowing that there is a container.

## Multiple package containers:

More complicated. We need to create a predictable name-space for the containers. What we do is `mulled-v2-`:``. Otherwise, same procedure as the single-package-containers. We use a build and a runtime container. We do *not* use a Dockerfile in both cases.

Try this one out to get a feeling how it works: http://biocontainers.pro/multi-package-containers/

Create multiple package containers with simple PRs like this one: https://github.com/BioContainers/multi-package-containers/pull/427

## Tooling

What happens underneath is documented here a little bit:
https://docs.galaxyproject.org/en/master/admin/mulled_containers.html#automatic-build-of-linux-containers

## Containers

Docker/rkt containers are here: https://quay.io/organization/biocontainers
Singularity container are here: https://depot.galaxyproject.org/singularity/

## Misc

More container engines can be easily added, indeed we have plans for swifter etc.
Creating the Docker/rkt containers happens during the CI step of Bioconda for Linux and are also pushed from within the CI.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。