hashgraph / hashgraph/solo-weaver
Story 5.1 — Implement external-files.yaml download with disk space check and halt/skip policy
- Dominant language
- Go
- Stars
- 3
- Forks
- 0
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 46
Description
**Epic:** #502 — Network Upgrade Workflow (Execute Phase)
Implement the `external-files.yaml` download/verify step in the `solo-provisioner-daemon` execute-phase logic.
**Scope (daemon runs in the frozen window, after the UC pod is gone):**
- `phase.download: prepare` entries — the UC already downloaded these to `data/upgrade/current/`; the daemon only **verifies presence + `algorithm`/`checksum`**.
- `phase.download: freeze` entries — the daemon **downloads** them now.
**Acceptance criteria:**
- Support `https://` and `s3://` URLs. All external-file URLs are assumed public/anonymous (no credentials required); the S3 fetcher may pick up ambient env/local credentials if present but must not require them.
- Before downloading each entry, determine remote size via HTTP `HEAD` (Content-Length) / S3 `HeadObject` and verify sufficient free space at BOTH the temp download path and the final destination.
- After download, verify `algorithm`+`checksum` against the file.
- **Uniform failure policy** across ALL failure modes (insufficient disk, network error, HTTP 4xx/5xx, checksum mismatch):
- **Required** entry (`optional` absent or `false`): **halt the upgrade** and emit the matching event (`FileDownloadFailed` / `FileHashMismatch`).
- **Optional** entry (`optional: true`): log a warning, emit `FileSkipped`, and continue processing remaining entries.
- Emit events as JSONL via `pkg/eventlog`; K8s event emission uses the shared recorder from #544.
_Reuse:_ `pkg/software/downloader.go` (HTTPS) as a base; add an S3 getter, checksum verification, and a free-space (statfs) helper.
---
## Archives support (added per HIP external-files spec update)
The `archives:` top-level key (parser: #1084) introduces archive-level download logic:
- Download the archive, verify archive-level `algorithm`+`checksum` before extracting anything
- Disk-space pre-check must cover archive download size **plus** total uncompressed size. Use the optional `size` field on the archive entry and each `contents[]` entry when available
- Support `tar.gz`, `tar`, `zip` formats
- Support `gcs://` scheme in addition to `https://` and `s3://` (mirrors state-sources bucket scheme set)
- New JSONL error classifications: `ArchiveDownloadFailed`, `ArchiveHashMismatch`
- `optional: true` applies to the archive as a whole — no partial-member success
Depends on: #1084 (parser)
Contributor guide
Assessment
This issue has not been assessed yet.