AllenInstitute / AllenInstitute/AllenSDK

hash out production conda environments

Open
#412 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

DevOps documentation noprogram
Dominant language
Jupyter Notebook
Stars
398
Forks
163
PR merge metrics
No merged PRs in 30d

Description

In order to run informatics modules in LIMS, we install the allensdk and other repositories into conda environments on our SAN. The intended workflow is roughly:

  1. a change is made to allensdk/internal (or some other repo)
  2. a bamboo plan runs tests
  3. when those tests pass, a repository maintainer triggers a bamboo deployment which updates the conda environment at a specified path (in practice the update removes and recreates the environment).

The executables table in lims records the path to the appropriate conda environment for each distinct executable.

This system has been put in place multiple times by different people for different projects and repositories. In order to make this situation more straightforward and easy to maintain, we should:

  • determine which conda environments are currently in use
  • determine which build and deployment plans manage them
  • determine who is in charge of each deployment plan
  • remove redundant environments
  • remove redundant build and deployment plans
  • ensure that we have enough build resources for these plans

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inventorying the conda environments currently referenced by the LIMS executables table, then map each environment to its Bamboo build and deployment plan and responsible maintainer. Review changes in allensdk/internal and the related repositories only as needed to identify managed environments. Done means the current environments, plans, owners, redundancies, and build-resource needs are documented and the redundant items are removed.

Written by the indexing model from the issue text.

Assessment

Domain
build-system, ci-cd, devops, infrastructure
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.