HarperFast / HarperFast/rocksdb-prebuilds

A partial asset upload leaves a public release that breaks rocksdb-js installs and does not self-heal

Open
#23 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
BitBake
Stars
1
Forks
1
Avg merge
6d 7h
Merged PRs (30d)
3

Description

## Summary

`softprops/action-gh-release` creates the release and its tag *first*, then uploads the asset files.
If the upload dies partway — a network blip, a runner death, one 502 on asset 7 of 10 — the result
is a **public release with a partial asset set**, and the pipeline then treats that version as
built.

## Why it breaks consumers

rocksdb-js does not tolerate a missing asset, and does not fall back to an older release:

- `scripts/init-rocksdb/get-prebuild.ts` sorts all releases by `semver.rcompare` and takes
`releases[0]` — the newest. There is no "try the previous release" path.
- `scripts/init-rocksdb/download-rocksdb.ts` filters that release's assets for the running
platform and, finding none, throws `Asset rocksdb---. not found`.

So one partial publish breaks `init-rocksdb` for every consumer on the affected platform, and it
keeps breaking them for as long as the release is the newest — a completed older release does not
rescue them.

## Why it does not self-heal

1. The `release` job fails, so a failure Slack card does post — this part is *not* silent.
2. But the release and tag are already public.
3. "Re-run failed jobs" stands down: the pre-publish check in
[#19](https://github.com/HarperFast/rocksdb-prebuilds/pull/19) sees the tag exists and skips
publishing, so the run goes green having repaired nothing.
4. Re-dispatching is refused by the `check` job — "Release already exists; choose a new revision" —
which has no answer for a stable version.
5. The nightly's `releases/latest` comparison now matches the published version, so it will not
rebuild.

Recovery is manual: delete the public release and rebuild all ten targets, or upload the missing
archives by hand.

## Why it is filed separately

Pre-existing and independent of [#21](https://github.com/HarperFast/rocksdb-prebuilds/pull/21),
which only reorders the release ahead of the S3 mirror — surfaced by the cross-model review of that
PR. Note the asymmetry this exposes: #21 deliberately makes the *S3 mirror* best-effort because
nothing reads it, whereas the *release assets* are what every consumer actually reads, so a partial
asset set is the one that matters.

## Options

- **Verify the asset set after publishing** and fail loudly with a message that names the repair —
cheap, and turns "silently newest-but-broken" into a clear instruction.
- **Make the publish idempotent so a re-run repairs it.** `softprops/action-gh-release` will
upload missing assets to an existing release, so the pre-publish guard needs to distinguish
*this* run's release from a genuine concurrent one — e.g. a run-id marker in the release body —
and complete the upload rather than standing down.
- **Publish as a draft, then flip to published** once all ten assets are uploaded, so an
incomplete release is never visible to the releases API. Cleanest semantically; needs a check
that `getPrebuild` ignores drafts (the GitHub API omits drafts for unauthenticated callers but
includes them for tokened ones, and rocksdb-js passes `GH_TOKEN` when set).
- Independently, rocksdb-js could fall back to the newest release that actually *has* the needed
asset. That is defence in depth on the consumer side and does not remove the need to stop
publishing partial releases.

## Location

`.github/workflows/build.yml` — the `release` job's `Create release` step.

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/build.yml and the release job’s Create release step, then read scripts/init-rocksdb/get-prebuild.ts and download-rocksdb.ts to understand how incomplete releases affect consumers. Compare the listed publishing options and verify the chosen behavior against partial uploads, reruns, and release selection; done means a failed asset upload cannot leave the newest usable release permanently broken.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, typescript
Domain
ci-cd, devops, release
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.