ESMCI / ESMCI/manage_externals

Add option for mirror repositories

Open
#116 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
10
Forks
12
PR merge metrics
No merged PRs in 30d

Description

The request is to add an optional mirror parameter for an external which would point to one or more additional repositories that are mirrors. Mirrors are considered to be equivalent to each other, so the checkout should success as long as at least one mirror is accessible.

The use case for this comes from the CMEPS project where we have had to create multiple Externals.Machine.cfg files - one for each of several machines. The reason for this is because one of the external repositories is private and so we had to create separate clones of the external repository and put it in shared spaces on the machine file systems.

For example:

*Externals.Stampede2.cfg*
```
[fv3gfs]
branch = dev/nems-benchmark-2.0
protocol = git
local_path = FV3GFS
repo_url = /work/06242/tg855414/stampede2/GITLOCAL/FV3
required = True
```
*Externals.Theia.cfg*
```
[fv3gfs]
branch = dev/nems-benchmark-2.0
protocol = git
local_path = FV3GFS
repo_url = /scratch4/NCEPDEV/nems/noscrub/Rocky.Dunlap/GITLOCAL/FV3
required = True
```

If we had a mirror option, it might look like this:
*Externals.cfg*
```
[fv3gfs]
branch = dev/nems-benchmark-2.0
protocol = git
local_path = FV3GFS
repo_url = https://some.repo.on.the.internet/myrepo
repo_mirror = /scratch4/NCEPDEV/nems/noscrub/Rocky.Dunlap/GITLOCAL/FV3
repo_mirror = /work/06242/tg855414/stampede2/GITLOCAL/FV3
required = True
```
When mirrors are present, each repo is attempted in order and fails gracefully until all of the mirrors have been exhausted.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how Externals.cfg entries are parsed and how checkout attempts repo_url. Review the mirror examples and verify the checkout flow tries each repo_mirror in order, continues after an inaccessible repository, and succeeds when any mirror is available. Confirm behavior with the project's existing tests, though no specific test file is named in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.