ceph / ceph/ceph-salt

Detect network flakiness before applying the Salt Formula

Open
#394 1 comment 0 reactions 0 assignees View on GitHub
validation/check
Dominant language
Python
Stars
36
Forks
20
PR merge metrics
No merged PRs in 30d

Description

`ceph-salt apply` is known to fail in odd ways when running in an environment with poor network connectivity. These failures can be especially vexxing if the network connections are flakey - i.e., they succeed on some attempts, and fail on others. In such cases, a user might reasonably think that the failure is due to a bug in ceph-salt.

For example:

* when an external time server is configured, and connectivity with that external time server is flakey, `ceph-salt apply` can fail
* when the container image path points to a remote registry, and connectivity with that registry is flakey, `ceph-salt apply` can fail.
* when ceph-salt attempts to use `zypper` to install packages on nodes, and connectivity with remote zypper repos is flakey, `ceph-salt apply` can fail.

It would be nice if we could detect network flakiness before starting to apply the Salt Formula. The purpose of this ticket is to collect ideas for how to do that.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the `ceph-salt apply` entry point and trace how the Salt Formula reaches the external time server, remote container registry, and zypper repositories. Review the issue's proposed detection ideas before choosing an approach; the work is done when a concrete pre-apply network-flakiness check is agreed and its behavior is validated for these failure sources.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.