vectordotdev / vectordotdev/vector
Migrate packages.timber.io artifacts to dd-cose-releases and retire obsolete content
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 22.6k
- Forks
- 2.3k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 146
Description
Goal
Migrate Vector release artifacts from packages.timber.io to the Datadog-owned dd-cose-releases bucket, move first-party consumers to the new endpoint, then retire obsolete content and eventually decommission the old hosting.
Current state
- Provision
dd-cose-releasesthrough Cloud Inventory with public object reads, encryption, logging, and prefix-scoped publisher/operator access (DataDog/cloud-inventory#68539). - Publish future stable, nightly, and custom builds to the
vector/prefix using GitHub OIDC (#26154). - Verify the custom-build workflow can publish to
vector/custom/and that the objects are publicly downloadable. - Migrate 106 immutable stable-version prefixes through
0.57.0: 2,952 objects totaling 108,325,104,776 bytes. All 1,636 non-empty source artifacts matched the destination by size; two broken zero-byte aliases whose redirect targets did not exist were intentionally omitted. - Stop publishing stable
.Xaliases to COSE (#26163). - Decide not to expose stable
vector/latest/orvector/latest.txtin COSE (#26168, merged into #26079). - Switch the website, installer, documentation, and Homebrew release tooling to exact-version COSE URLs and publish the deprecation notice (#26079).
- Dual-publish the next stable release to both endpoints and verify its checksums and public downloads.
- After that release, stop all new Vector writes to
packages.timber.io. - Delete legacy nightlies older than seven days while preserving the rolling seven-day window until the old hosting is retired.
- Complete consumer and retention checks for old custom builds, legacy Timber products, obsolete Helm content, and the remaining frozen stable objects before deletion.
Decisions
dd-cose-releasesis the COSE-owned shared release-artifact bucket. Vector owns thevector/prefix; Quickwit can later use an independently permissionedquickwit/prefix.- The supported public artifact root is
https://dd-cose-releases.s3.amazonaws.com/vector/. Exact object URLs are public; bucket and prefix listing is intentionally disabled. - Stable COSE artifacts use only immutable
vector/<exact-version>/prefixes. COSE does not provide stablelatest,latest.txt, minor.X, or major.Xaliases. - GitHub Releases continue receiving stable assets as an additional mirror and release-discovery surface, but first-party direct downloads use COSE.
- The next stable release is dual-published to COSE and
packages.timber.io. Later releases publish only to COSE. - Existing
packages.timber.ioURLs remain available during the deprecation window. After writes stop, retained objects remain frozen until their announced deletion/decommission date. - Historical nightly and custom objects are not migrated. Future nightlies publish under
vector/nightly/<date>/andvector/nightly/latest/; future custom builds publish undervector/custom/. - Old-bucket nightlies older than seven days are deleted. Old custom builds are deleted later after retention and consumer checks.
- Helm chart packages remain GitHub Release assets;
helm.vector.devremains the repository-index endpoint. - Legacy Timber product artifacts (
agent/,cli/, andcloudwatch-logs-lambda-function/) and obsolete Helm prefixes are not migrated and are deleted only after their own consumer and retention checks.
Remaining plan
- Merge #26079 and publish the user-facing deprecation announcement in Vector
0.58.0. - Dual-publish and verify the next stable release at both endpoints.
- Stop all subsequent Vector writes to
packages.timber.iowhile leaving retained objects readable and frozen. - Add automated seven-day retention for old-bucket nightlies.
- Announce the final retention/decommission date after consumer checks, then delete the approved legacy content and decommission the old hosting.
- Add Quickwit later with a separate publisher identity and
quickwit/prefix; neither project receives write access to the other's prefix.
Acceptance criteria
- Exact-version Vector artifacts are publicly downloadable from
dd-cose-releases/vector/and first-party stable-download consumers use that endpoint. - The next stable Vector release is published and checksum-verified at both the old and new endpoints.
- Future nightly and custom builds publish only to COSE; stable builds publish only to exact-version COSE prefixes after the one-release transition.
- After the transition release, no new Vector artifacts are written to
packages.timber.io. - GitHub Releases continue receiving stable release assets.
- The deprecation entry identifies the replacement URL, migration guidance, old-URL availability, and retirement boundaries.
Pending decision
- Final retention/deletion date for frozen
packages.timber.iostable objects and the remaining approved legacy content.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the remaining plan and PR #26079, then inspect the release-publishing workflows introduced by #26154 and the existing stable, nightly, and custom consumers. Verify the next stable release at both endpoints, switch first-party downloads to exact-version COSE URLs, and confirm retention, deprecation, and no-new-write acceptance criteria before retiring approved legacy content.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, github-actions
- Domain
- cloud, devops, release
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100