devantler-tech / devantler-tech/ksail

ci: harden the release pipeline against transient network flakes (retry apt + goreleaser downloads)

Open
#5,954 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
165
Forks
12
Avg merge
5h 41m
Merged PRs (30d)
337

Description

> 🤖 Generated by the Daily AI Assistant

## Problem

The tag-triggered release pipeline (`cd.yaml`) fails frequently on **transient network flakes** in un-retried fetch steps, and each failure blocks the whole release — no GitHub release object or binaries get published, which in turn breaks the Homebrew cask bumps (`Audit Casks` → 404 on the missing assets) and stalls `brew upgrade ksail` for users (the cascade context of #5875).

**Evidence — v7.159.0 (CD run 28899254339) failed across *four* attempts on two different transient causes:**
- `🧩 Release Desktop App` → *Setup desktop build* → `apt-get` from `packages.microsoft.com`: `Clearsigned file isn't valid, got 'NOSPLIT'` (exit 100) — a Microsoft-apt CDN hiccup.
- `🧩 Release CLI` → GoReleaser pre-hook (`go`): `failed to download CLI binary: failed to save tarball: stream error: stream ID 1; INTERNAL_ERROR; received from peer` — a `proxy.golang.org` HTTP/2 flake.

None of these are code defects — they are single-attempt network fetches in a ~32-minute job, so any one hiccup fails the entire release and leaves the tag published but the release absent.

## Proposed direction

Add **bounded, transient-only retries** to the network-fragile steps of the release path so a single CDN/proxy hiccup self-heals instead of failing the release:
- **apt fetches** in the `setup-desktop-build` composite action → `apt-get -o Acquire::Retries=3 …` (and/or a retry wrapper).
- **GoReleaser `go` pre-hook downloads** (both the CLI and desktop goreleaser invocations) → wrap the goreleaser step in a bounded retry (e.g. a `retry`-action, kept pinned by SHA) and/or harden `GOPROXY`/`GOFLAGS` resilience.
- Keep retries **small and bounded** and scoped to network errors so genuine build/signing failures still fail fast (don't mask real regressions).

Operational note worth capturing alongside the fix: a failed publish leaves **no release object at all** (not even a draft), so recovery requires a **full** `gh run rerun ` — a partial `--failed` rerun can't reconstruct the ephemeral draft-release state and fails with `release not found`. A self-healing bounded retry would remove most of these manual reruns.

## Rough size

**S–M** — workflow + composite-action YAML only, no Go code. Validatable via actionlint/zizmor; the behavioural proof is the next clean release.

Contributor guide

Open the contributing guide

Research direction

Start with cd.yaml and the setup-desktop-build composite action, then inspect the CLI and desktop GoReleaser steps that perform downloads. Run actionlint and zizmor before testing the workflow; done means bounded retries cover transient apt and download failures while genuine build or signing failures still fail promptly.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, go
Domain
ci-cd, devops, release
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
67/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.