improve dependency installation on cloud
- Dominant language
- R
- Stars
- 10
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
the google life sciences api creates a new snakemake container to run each of our jobs
so our dependencies must be installed anew by snakemake every time it runs a job, since each job is run in a new, empty container
this process may add substantially to the runtime of some of our instances and the cost of using them
we have two options:
1) separate our dependencies into different env files to install only the necessary dependencies for each job
2) create a custom snakemake container with our dependencies already installed in it and force snakemake to use our container instead of its own
Contributor guide
No contributing guide indexed for this repository
Research direction
No files or tests are named. Start by locating the Snakemake configuration for Google Life Sciences jobs and its dependency environment or container settings. Compare the two proposed approaches, then consider the work done when jobs no longer reinstall unnecessary dependencies or use a custom container with the required dependencies preinstalled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud
- Domain
- bioinformatics, cloud, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100