hashgraph / hashgraph/solo-weaver
feat(manifests): add archives: support to external-files.yaml parser
- Dominant language
- Go
- Stars
- 3
- Forks
- 0
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 47
Description
**Epic:** #502 — Network Upgrade Workflow (Execute Phase)
**Parent:** #408
## Context
The HIP external-files spec now includes an `archives:` top-level key (sibling to `files:`) for verifiable archive download and extraction. The motivating case is TSS WRAPS proving keys (~2 GB `wraps-v1.0.0.tar.gz` containing 4 members) that must be staged atomically before the node starts.
This story adds the parser and validation support in `pkg/manifests`. Download and install execution are handled by #537 and #538 respectively (updated to include archives).
## Scope
### New types
- `Archive` struct: `url`, `format`, `algorithm`, `checksum`, `contentType`, `destination`, `optional`, `phase`, `contents`
- `ArchiveMember` struct: `path`, `algorithm`, `checksum`, `size` (optional), `destination` (optional override)
- `ArchiveFormat` enum: `tar.gz`, `tar`, `zip` — explicit, never inferred from URL
### Add `Archives` field to `ExternalFiles`
```go
type ExternalFiles struct {
Header `yaml:",inline"`
Files []ExternalFile `yaml:"files,omitempty"`
Archives []Archive `yaml:"archives,omitempty"`
}
```
### Parse-time validation rules
- `url`, `format`, `algorithm`, `checksum`, `destination`, `phase.download`, `phase.install` are required and non-empty
- `format` must be one of `tar.gz | tar | zip`
- `url` must use a recognised scheme (`https://`, `s3://`, `gcs://`)
- `destination` must begin with a recognised marker prefix followed by `/` (same closed set as `files[]`)
- `contents` is required and must declare at least one entry
- `contents[].path` must be relative: no leading `/`, no `.` or `..` segment, no drive letter, no backslash separator. Paths must be unique within a single `archives[]` entry
- `contents[].algorithm` and `contents[].checksum` are required and non-empty
- **Global destination uniqueness**: every resolved install path — across all `archives[]` members and all `files[]` entries — must be unique
### URL scheme validation for `files[]`
While here, add URL scheme validation to `files[]` entries as well. The HIP specifies `https://`, `s3://`, `gcs://` as the allowed set. Currently `files[]` only checks non-empty. Add an `allowedURLSchemes` closed set and accessor (following the `allowedBucketSchemes` / `allowedDestinationPrefixes` pattern).
### Tests
- Happy-path archive with all fields populated
- Archive with optional per-member `destination` override
- Validation failures for each rule above
- Global destination uniqueness across files + archive members
- Path safety: reject `..`, leading `/`, backslash
- Lenient: unknown fields in archives silently ignored (HIP-1494)
## Acceptance Criteria
- [ ] `ParseExternalFiles` accepts manifests with `archives:` entries
- [ ] All parse-time validation rules enforced with `ValidationError`
- [ ] URL scheme validation added to both `files[]` and `archives[]`
- [ ] Global destination uniqueness across files and archive members
- [ ] Path safety enforced on `contents[].path`
- [ ] Tests cover happy path, all validation failures, and lenient parsing
Contributor guide
Research direction
Start at ParseExternalFiles in pkg/manifests and read the existing allowedBucketSchemes and allowedDestinationPrefixes patterns. Run the existing manifest parser tests, then use the listed archive, URL, path-safety, uniqueness, and lenient-parsing cases as coverage; done means all acceptance criteria pass with ValidationError for invalid manifests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100