trailofbits / trailofbits/coop

Research: hardening gaps in the coop update / install verification chain

Open
#422 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement security
Dominant language
Rust
Stars
243
Forks
13
Avg merge
1d 20h
Merged PRs (30d)
30

Description

Status: research only

This is an audit write-up, not a proposal we have committed to. No decisions have
been made about implementing any of it.
The intent is to record what the current
coop update / install.sh verification chain does and does not guarantee, so the
items below can be triaged onto the roadmap — or explicitly declined and recorded as
accepted trade-offs — rather than rediscovered later.

Nothing here reports an exploited or practically-exploitable weakness. Every item
requires an attacker who already controls the network path, the release CDN, the
GitHub repository, or the local process environment. Several are already documented
as known limits in docs/trust-model.md.

The review was structured around the threat taxonomy from
The Update Framework — rollback,
freeze, endless-data, mix-and-match, wrong-software-installation, key compromise.
Adopting TUF itself is not proposed (see Not proposed); it was
useful only as a checklist.

All line references are against main at time of writing.

What the chain does today

coop update (src/update.rs) and install.sh both:

  1. Fetch release metadata from the pinned trailofbits/coop repo (compile-time const).
  2. Validate a user-supplied --version as semver before it enters the API URL path
    (normalize_tag, path-traversal guard).
  3. Download the tarball and SHA256SUMS, and verify the tarball against it.
  4. Run gh attestation verify --repo trailofbits/coop for Sigstore/SLSA build
    provenance — best effort, skipped when gh is absent.
  5. Extract with tar --no-same-owner --no-same-permissions, then atomically
    rename over the running binary.

What already holds up well

Worth stating explicitly, because it is the property this kind of audit most often
finds missing: coop is structurally immune to mix-and-match, where a client is
served a combination of artifacts that never coexisted on the server. A release is a
single self-contained tarball per target
(.github/workflows/release.yml:64-73), so there is no multi-artifact consistency
problem to get wrong and no need for consistency metadata to solve it. That property
is worth preserving deliberately as more binaries ship in a release — the
coop-proxy work in #411 keeps it by placing the proxy inside the same tarball
rather than publishing it as a second asset, which is the right instinct.

The --version path-traversal guard, the exact-filename match in
parse_sha256sums, and the constant-size Sha256Hash comparison are all sound.

Findings

1. Verification can degrade to nothing

verify_attestation returns Ok(()) with only a tracing::info! when gh is
absent (src/update.rs:394-404); install.sh:113-126 does the same. The remaining
check is SHA256SUMS — fetched from the same release, over the same channel,
unsigned (src/update.rs:542).

An unsigned checksum served alongside the artifact it describes is an integrity check
against corruption and truncation, not an authenticity check: anyone able to serve
the tarball can serve matching sums. docs/trust-model.md:126 onward describes the
checksum as "the floor," which is accurate as written but reads stronger than it is.

install.sh has a second, worse case: with no sha256sum or shasum on the host,
the checksum is also skipped with a warning and return 0
(install.sh:96-111, skip at :104). A host with neither gh nor a SHA-256 utility
installs entirely unverified. macOS ships shasum, but minimal container images may
ship neither.

Highest-value item of the set. Options in Closing finding 1.

2. No freshness bound — a held-back client cannot tell

maybe_run_background_check stamps last_checked_at = now before spawning the
fetch thread (src/update.rs:706-719, stamp at :714), and the thread only writes
latest_known_version on success. A network that always fails therefore advances the
timestamp indefinitely while the known-latest version stays frozen, and no surface
ever reports "we have not successfully reached GitHub in N days."

Related: an attacker who serves an older latest produces "Already on latest,"
indistinguishable from the truth. No high-water mark of the highest version ever seen
is retained, so metadata rollback is undetectable.

Neither is a code-execution risk on its own — the failure mode is a client sitting on
a known-vulnerable version while believing it is current.

Cheap to close: a separate last_success_at field plus a staleness warning, and a
highest_seen_version field plus a warning when the server reports lower. Both are
local state, no protocol change, roughly 15-25 lines.

3. No size or duration bounds on downloads

curl -fsSL is used with no --max-filesize, --max-time, or
--speed-limit/--speed-time (src/update.rs:268-284, src/update.rs:308-323, and
the equivalents in install.sh). A malicious or malfunctioning endpoint can return an
unbounded stream, or trickle bytes indefinitely and hang the update.

GitHub's release JSON carries a per-asset size, which Asset
(src/update.rs:160-165) does not parse. Using it as the cap, bounded by an absolute
ceiling so attacker-controlled metadata cannot authorize an arbitrarily large
transfer, would close this.

4. No protocol pinning across redirects

-L is passed without --proto '=https' --proto-redir '=https', and
browser_download_url comes from the response body straight into curl's argv
(src/update.rs:162src/update.rs:308-323). Under a compromised-metadata threat
model that is an attacker-chosen URL, including plaintext http:// or file://. A
one-flag change on each curl invocation, optionally plus a host allowlist when the
API base is not overridden.

5. An environment variable relocates the trust anchor in release builds

COOP_UPDATE_API_BASE_URL redirects the update origin and disables attestation
verification (src/update.rs:112, :120-127, :391-393) in stock release binaries,
with a tracing::warn! to stderr as the only signal. Anything able to set process
environment — a shell rc, a CI wrapper, direnv reading an .envrc from a repository
you cd into — silently redirects the next coop update to an unverified origin.

docs/trust-model.md already records this as a known test-only mode. The open
question is whether it should be compiled out of default release builds rather than
gated at runtime. build.rs already bakes COOP_BUILD_KIND from
COOP_FORCE_BUILD_KIND, and tests/integration-update.sh:128 relies on that
mechanism, so the same approach extends to this override without losing test coverage.

Lower priority
  • Installed version is never checked against the release tag. perform_update
    (src/update.rs:517) verifies bytes and build provenance but never confirms the
    extracted binary reports the version it was published as. Attestation binds bytes
    and build origin, not version. More a release-process robustness gap than an
    attack; the fix is a few lines.
  • install.sh matches the checksum line by unanchored substring.
    install.sh:148 uses grep "${TARBALL}" "${TMPDIR}/SHA256SUMS" | cut -d' ' -f1,
    where parse_sha256sums in update.rs matches the filename exactly. Fail-closed
    today and no asset name is a substring of another, so this is parity and
    brittleness rather than a hole.

Closing finding 1: two options

Option A — verify the Sigstore bundle in-process. #421 publishes the provenance
bundle as an attestations.jsonl release asset and verifies it with
gh attestation verify --bundle, which makes no API call and needs no credential.
With that asset published, a client-side verifier (e.g. the sigstore crate) could
read the same bundle directly, which removes gh from the trust path and lets
verification become mandatory. No key custody, and no release-workflow change beyond
what #421 already does. It does not change the trust root: Sigstore attests "built by
a workflow in trailofbits/coop," so an attacker with repository write access still
obtains a valid attestation. Cost is a substantial new dependency.

Option B — sign SHA256SUMS with an offline key and embed the public key in the
binary.
Also makes verification mandatory and gh-independent, and additionally
survives GitHub compromise, because the key lives outside GitHub. Needs a signature
dependency (Cargo.toml:21 already pulls sha2 for Sha256Hash, but nothing that
verifies signatures) — a small, well-scoped one such as ed25519-dalek, notably
lighter than Option A's.

The real cost of Option B is key lifecycle, and it should not be started without a
decision on that. A single unrotatable embedded key would be a worse position than
today: losing it means being unable to ship, with no revocation path. If we pursue B,
the lifecycle needs a threshold of keys rather than one, an expiry baked into the
trust anchor, and a rotation procedure written into RELEASING.md before the first
key is generated.

A and B are not exclusive and could land in either order. A is cheaper and closes
"verification silently becomes a no-op." B additionally closes "our own CI is a single
point of compromise." They address different threats.

Relationship to #421

#421 is a prerequisite for finding 1, not a fix for it. Before #421, attestation
verification required a GitHub credential authorized for the trailofbits org, so it
could never have been made mandatory — external users would have been locked out.
That is the bug #421 fixes, and it is what makes Option A cheap. It does not change
the gh-absent skip, the unsigned checksum floor, or the install.sh
no-SHA-256-utility path.

#421 also inherits findings 3 and 4 in its new bundle download
(fetch_attestation_bundledownload_asset), and extends the finding-5 override
to a second code path. Neither is a regression — the new code follows the conventions
of the code around it — but both are worth noting so the fixes, if pursued, cover the
new call site too.

Separately, and independent of everything above: update.rs and install.sh disagree
about fallback in #421. update.rs checks for the asset before downloading, so a
missing asset falls back to the API while a failed download fails the update closed.
install.sh conflates the two and falls back to the API on either. The consequence is
mild — the API path still verifies, so selectively dropping the bundle asset denies
updates to users without an SSO session rather than bypassing verification — but the
asymmetry is undocumented and probably wants to be deliberate.

Suggested sequencing, if we pursue any of this

  1. Findings 3, 4, 5 — one small PR. No design decisions, no new dependencies, and
    softens nothing in the existing chain.
  2. Finding 2 — separate PR, local state only.
  3. Finding 1 — needs an Option A / Option B decision first. Option B additionally
    needs a key-custody decision, which is operational rather than technical.

The lower-priority items are good-first-issue material and can go any time.

Not proposed

Adopting TUF itself. Its machinery — four metadata roles, snapshot/timestamp
re-signing, delegation, a repository service holding an online timestamp key — is
built for registries with many independent publishers and mirrors. coop is one binary,
one publisher, three target triples, hosted on GitHub Releases. Running a metadata
service to protect a single artifact while still depending on GitHub to host it is
standing operational cost that will rot, because nothing else would depend on it. The
three properties worth taking are freshness state, transfer bounds, and a trust anchor
that is not relocatable — all small local changes, not a framework.

The GHCR devcontainer-feature pulls (docs/trust-model.md:62) are further out of
scope again: those install snippets execute in the guest, where the VM is the blast
radius.

References

  • src/update.rs, install.sh, .github/workflows/release.yml
  • docs/trust-model.md — the "coop update trust chain" section (:126), which
    already documents the best-effort attestation and the COOP_UPDATE_API_BASE_URL
    trade-off
  • #421 — offline bundle verification

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/update.rs and install.sh, then compare the documented limits in docs/trust-model.md and the build override coverage in tests/integration-update.sh. First identify which finding or signing option has maintainer agreement; done means a concrete, bounded follow-up is selected or the trade-off is explicitly recorded, with affected verification paths and tests identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, shell
Domain
cli, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.