Git object-store A3 conformance probe fails on GCS S3-interop (`if_match_race`) — consider a native GCS backend
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
## Environment
- Relay: `ghcr.io/block/buzz:sha-2d26db6` (`2d26db6d8a7381ff0dc9b50194fd28d42957de74`); reproduced identically on `sha-acfbb1b` (2026-07-22 main)
- Object storage: **Google Cloud Storage via the S3-interop XML API** (`BUZZ_S3_ENDPOINT=https://storage.googleapis.com`, HMAC service-account credentials)
- Deployment: GKE, chart `buzz` 0.1.6, external Postgres/Redis
## What happens
Relay startup is fatal at the A3 git conformance gate:
```
INFO running git object-store conformance probe (A3 gate)
Error: git conformance probe failed: conformance probe failed in phase 'if_match_race' (round 0, key probe/pointer-0d592977-…)
```
Media paths on the same bucket/credentials work (`Media storage connected`;
put/get verified end-to-end through the app).
## Why
The probe is right to fail: GCS's S3-interoperability layer does not honor
S3-style conditional writes (`If-Match`/`If-None-Match` preconditions on PUT),
so the linearizable pointer CAS that `run_conformance_probe` verifies (and
that the manifest-pointer protocol needs) cannot be satisfied. This makes
git-on-object-storage unusable on GCS for any deployer using the documented
S3 configuration — the probe converts that into a hard startup failure.
## What we did (and why it's not a fix)
Booted the pilot with `BUZZ_GIT_CONFORMANCE_PROBE=false` and quarantined git
features operationally. That's an availability workaround, not a solution —
the probe's verdict about pointer CAS still stands.
## Proposal
GCS *does* support real CAS — via its native preconditions
(`x-goog-if-generation-match: 0` for create-if-absent, generation-match for
compare-and-swap). Options in rough order of preference:
1. **Native GCS storage backend** for the git object store (JSON/XML API with
generation preconditions) behind the existing storage trait.
2. **Conditional-write shim for GCS endpoints**: detect
`storage.googleapis.com` and translate the pointer-CAS operations to
`x-goog-*` preconditions while leaving plain put/get on the S3 path.
3. At minimum, **document** that git-on-object-storage requires a backend with
S3 conditional-write semantics (AWS S3 ≥ 2024 If-Match support, MinIO, R2?)
and name GCS S3-interop as unsupported — so deployers hit a doc instead of
a fatal probe.
We (Apiary — hosted Buzz on GCP) are happy to test any of these against real
GCS, and would rather contribute the backend than run a MinIO shim in front
of GCS.
Contributor guide
Research direction
Start with the existing storage trait and the run_conformance_probe entry point, then review how the if_match_race phase performs pointer CAS. Confirm the selected GCS approach against the native generation preconditions and the documented S3 configuration. Done means GCS git object storage passes the conformance gate, or its unsupported status is clearly documented without requiring a fatal startup failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, rust
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100