Imageomics / Imageomics/Imageomics-guide

Advise on HPC conda environments consuming home directory quota

Open
#30 1 comment 0 reactions 0 assignees View on GitHub
documentation question
Dominant language
Python
Stars
6
Forks
0
Avg merge
1h 28m
Merged PRs (30d)
3

Description

It's been a repeated occurrence that conda environments, which default to living in a user's home directory, causes the user's quota to exceed the file count limit on OSC.

To fix this, conda can be configured to store environments and packages in a custom location, scratch storage being an appropriate option I use and have recommended. E.g., in a `~/.condarc` file:

```bash
solver: libmamba
envs_dirs:
- /fs/scratch/PAS2136/$USER/conda-envs
pkgs_dirs:
- /fs/scratch/PAS2136/$USER/conda-pkgs

channels:
- conda-forge
- defaults

```

Would it make sense to do the following:
- Add a general discussion of this issue under the [Conda section of Virtual Environments](https://imageomics.github.io/Imageomics-guide/wiki-guide/Virtual-Environments/#conda-environments)
- Provide a generic `.condarc` template that can be copied using `/path/to/desired/user/conda-envs` and `/path/to/desired/user/conda-pkgs`

Should an entry be made into the internal guidelines wiki for Imageomics using the actual paths?

Contributor guide

Open the contributing guide

Research direction

Start with the Conda section of Virtual Environments linked in the issue and review the repository's existing documentation structure. Add the general quota guidance and a copyable generic .condarc template, then clarify whether the internal Imageomics wiki should document the actual paths; done means both audiences have appropriate instructions.

Written by the indexing model from the issue text.

Assessment

Tech stack
anaconda
Domain
documentation, hpc
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.