hiero-ledger / hiero-ledger/hiero-consensus-node

[Investigation] Clarify how adhoc builds (002) reach the migration pipeline

Open
#27,176 0 comments 0 reactions 1 assignee Claimed by @imalygin View on GitHub
Dominant language
Java
Stars
406
Forks
226
Avg merge
3d 4h
Merged PRs (30d)
210

Description

## Summary

Adhoc (`build--.zip`) and release (`build-v.zip`) artifacts share the same flat namespace in the **internal** release-artifact bucket. This co-location is an accidental side-effect of a shared upload step, not a deliberate design. This ticket proposes separating them.

Two options are on the table — a dedicated adhoc bucket vs. an `adhoc/` directory in the existing bucket. The final choice is DevOps's, with a lean toward the directory as the least-intrusive option.

## Background / current behavior

Image

- `002: [USER] Deploy Adhoc Artifact` dispatches `850-call-build-release-artifact.yaml` with `version-policy: branch-commit`, passing `bucket-name = RELEASE_ARTIFACT_BUCKET_NAME`.
- The standard tag-triggered production release (`301`) dispatches the same `850` with `version-policy: specified`.
- Both paths share `850`'s single artifact-upload step. This split has been the status quo for a long time, per DevOps requirements.

## Adhoc / release artifact mixing in the internal bucket

### Root cause

Both `002` (adhoc) and the release path call the same `850`, whose "Upload Artifacts (DevOps GCP Bucket)" step writes to the bucket **root** (`destination: ${{ secrets.bucket-name }}`, `parent: false`) for every policy except dry-run. So adhoc `build--.zip` and release `build-v.zip` land in the same flat prefix. Nobody chose to co-locate them; adhoc rides the shared step.

### Two useful facts for the fix

- **The bucket choice lives in `002`, not `850`.** `850` is bucket-agnostic — it writes to whatever `secrets.bucket-name` it is handed, and `002` currently hands it `RELEASE_ARTIFACT_BUCKET_NAME`.
- **Adhoc and release artifacts are name-distinct and read by different consumers.** `223-disp-sdct-controller.yaml` (SDCT) reads `build--.zip`; `851-call-deploy-preview.yaml` (preview deploy) validates `build-v.zip`. Relocating adhoc artifacts affects only the branch-commit consumers.

### Options

Persisting adhoc builds is a requirement, so both options keep the builds stored — they differ only in *where*. The release path (`301` → `build-v.zip` at root, `851`, release Jenkins) is untouched in both.

| Dimension | Option A — Dedicated adhoc bucket | Option B — `adhoc/` directory (existing bucket) |
|---|---|---|
| Write-side change | One line in `002` — swap `bucket-name` to a new `ADHOC_ARTIFACT_BUCKET_NAME` secret; `850` untouched | Policy-aware `destination` in `850` (`branch-commit` -> `/adhoc/`, `specified` -> root) |
| New infrastructure | New GCS bucket to provision + own | None |
| IAM / access boundary | Separate — permissions independent of releases | Shared — bucket-level IAM applies to both |
| Retention / lifecycle | Independent bucket policy | Prefix-scoped rule (`matchesPrefix: adhoc/`); else inherits bucket default |
| Consumers to repoint | Adhoc readers switch bucket | Adhoc readers switch to `adhoc/` prefix (same set) |
| Reversibility | Easy (point back) | Easy (revert destination) |

Notes for the decision:

- **Consumer-repointing effort is the same either way** — adhoc readers change the `gs://` URI they read from regardless. "Least intrusive" is about infrastructure overhead, where Option B wins.
- **Retention is not a differentiator** — GCS lifecycle rules support prefix matching, so `adhoc/` can have its own expiry without a separate bucket.
- **The real distinguisher is IAM** — GCS access control is bucket-level, so Option B shares a permission boundary between adhoc and release. Option A is warranted if DevOps wants a hard access boundary (or independent ownership/quota) for adhoc builds.
**Proposal:** Option B (`adhoc/` directory) as the default — lower overhead, retention still scopable per-prefix — with Option A as the alternative if DevOps prefers the IAM boundary. Final call is DevOps's.

**Safety constraint (both options):** the change sits on the shared `850` path (or `002`'s secret wiring). Do not disturb where release `build-v.zip` lands — `851` and release Jenkins expect it at the bucket root.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.