ESCOMP / ESCOMP/CTSM

ctsm_pylib container?

Open
#3,239 3 comments 1 reaction 0 assignees View on GitHub
b4b devops enhancement priority: low
Dominant language
Fortran
Stars
352
Forks
361
Avg merge
2d 21h
Merged PRs (30d)
7

Description

We have a few GitHub workflow jobs that require the use of `ctsm_pylib`. Whenever those run, they have to download the packages and install the environment—there's no way to cache it. This counts towards the ESCOMP org's bandwidth billing.

It would be cheaper (and maybe faster) to have a `ctsm_pylib` container that the jobs could run in. As far as I can tell, as long as that container is public and hosted on GitHub Container Registry (as our `ctsm-docs` container is), this would not count toward our bandwidth usage.

On the other hand, if we're close to having self-hosted GitHub runners, it's possible the `ctsm_pylib` environment could be cached on those, avoiding this issue—at least for people with permissions to use those runners.
___
I've had a stab at this, the results of which you can see on the PR at samsrabin/CTSM#13.

The container and Python unit testing work on my Mac. From the top level of a CTSM checkout, open an interactive terminal in the container like so:
```shell
podman run --rm -it -v $PWD:/home/mambauser ghcr.io/samsrabin/ctsm/ctsm_pylib-container:latest bash
```
Then, in the container, do:
```shell
cd python
./run_ctsm_py_tests -u
```
and it should work. (Although depending on your CTSM version you may have some failing/erroring tests.)

However, I can't get it to work in the GitHub workflow. Once that's working, it'd be nice to try minimizing the image size using Alpine Linux instead of what comes with the default micromamba image (I think Arch). We would also want to have a dual arm64/amd64 image as well; the current one (`dd25c67e7ed6`) is just arm64.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.