devantler-tech / devantler-tech/ksail

System-test install retries cannot recover an already-created Helm release

Open
#6,960 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
165
Forks
12
Avg merge
5h 41m
Merged PRs (30d)
337

Description

> 🤖 Generated by the Agentic Engineer

## Evidence and reproduction

The system-test action retries `ksail workload install` three times with the same Helm release name. If the first install creates a release and then times out waiting for its Deployment, later attempts immediately fail with `cannot reuse a name that is still in use`. There is no release recovery between attempts.

Observed in [KSail PR #6930, job 101956283664](https://github.com/devantler-tech/ksail/actions/runs/34192303845/job/101956283664) at `9f4f94c85e012e1578c3abe4d17c69ee7343f610`: attempt 1 timed out at 06:26:26 UTC; attempts 2 and 3 hit the reserved name at 06:26:37 and 06:26:47. Kubernetes events identify a pod-sandbox timeout and reserved sandbox name. That initial infrastructure failure is separate from the deterministic retry defect.

The same loop remains in `.github/actions/ksail-test-workload/action.yaml:104-124` on main `7e35050bf962034a6b10033b070636dbc6a59553`. Running that exact shell block with an install stub that creates release state before returning a readiness error reproduces all three failures and exit 1.

## Impact and expected behavior

This affects contributors whose system tests encounter a readiness failure after Helm records a release. The advertised retry cannot recover, even if the original infrastructure problem clears. Retries should either safely recover only the test-owned release before reinstalling, or resume its readiness check without bypassing the install behavior under test.

## Acceptance criteria

- [ ] A deterministic regression starts with a failed install that has already created the test release and proves a later attempt can succeed after recovery.
- [ ] Persistent install/readiness failures still fail the test, and cleanup/recovery failure is not silently ignored.
- [ ] Recovery is restricted to the test-owned release and namespace; no unrelated release or resource is removed.
- [ ] Success still exercises `ksail workload install` and verifies both deployment readiness and the Helm release.
- [ ] The affected hosted system test passes with the repaired action.

Rough size: small. The later diagnostics-upload cancellation is separate; it is not evidence that the original test passed.

Contributor guide

Open the contributing guide

Research direction

Start with the retry loop in .github/actions/ksail-test-workload/action.yaml:104-124 and reproduce the documented shell block using an install stub that leaves release state before returning a readiness error. Verify that a later attempt recovers only the test-owned release and namespace, succeeds through ksail workload install, checks deployment and Helm-release readiness, and still fails visibly when recovery or installation remains unsuccessful.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, kubernetes
Domain
ci-cd, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.