galaxyproject / galaxyproject/galaxy

Improve Dependency Reproducibility Robustness

Open
#4,191 0 comments 3 reactions 0 assignees View on GitHub
kind/enhancement
Dominant language
Python
Stars
1.9k
Forks
1.2k
Avg merge
2d 23h
Merged PRs (30d)
158

Description

In four sections going from "we are asking for disaster if don't do this" to "wow, that would be pretty cool".

For the community on depot for we should back up all:

- [ ] Conda artifacts used by known tools. https://github.com/maxpoint/conda-mirror
- [ ] Quay.io containers under the biocontainers namespace.
- [ ] Singularity containers matching the biocontainers.

On Galaxy instances we should record:

- [ ] Record the full list of packages (including versions and sources) used in Conda for a given job.
- [ ] Record every container identifier and some sort of hash.
- [ ] Provide instructions and potentially infrastructure for local Docker repositories to store and serve containers.

On Galaxy instances we should provide the option to backup:

- [ ] All Conda packages used for every job.
- [ ] All containers used for every job.

On Galaxy instances we should provide APIs for:

- [ ] Fetching commands and package lists for rebuilding Conda environments for given jobs outside of Galaxy.
- [ ] Downloading the container used for a given job.
- [ ] Build containers with packages matching those used natively for non-container jobs.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are identified. Start by breaking the four sections into separately scoped work and reading the linked conda-mirror project, then define which checklist item is being implemented and how backup, recording, or API behavior would be verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.