source-cooperative / source-cooperative/data.source.coop
Support standard SigV4 presigned URLs (inbound query-string auth)
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 24
- Forks
- 6
- Avg merge
- 1h 32m
- Merged PRs (30d)
- 1
Description
Summary
Support standard AWS SigV4 presigned URLs against data.source.coop so the frontend can hand a flat, self-authorizing GET URL to data-viewer iframes (COG/GeoTIFF, PMTiles, GeoParquet, …) — including for private/restricted products — without embedding the user's session credentials in the iframe.
Presigned URLs point back at the proxy (not the raw S3/Azure/GCS bucket), so the worker still performs the backend fetch and streams the bytes. This preserves per-download byte metering and the live location broadcast (a hard requirement) — only identity resolution changes, not the data path.
Decisions (scoping)
- Standard SigV4 presigned URLs, verified against the existing STS stack — not a bespoke token scheme. Stays interoperable with
awsCLI / GDAL/vsis3// SDKs and reuses the STS investment. - Backend credentials for private data are a separate workstream. This work assumes the proxy can already read non-public backends (today
registry.rs:188hardcodesskip_signature=true/ anonymous andDataConnectionDetailscarries no credential fields). This issue covers only the presigned-URL minting + verification layer. - GET/HEAD only (proxy is read-only; writes → 405). No presigned uploads.
The crux: inbound presigned (query-string) SigV4 verification
The core gap lives in the multistore crate (co-maintained). multistore-0.4.0/src/auth/identity.rs::resolve_identity returns Anonymous whenever there's no Authorization header, and reads the session token / payload hash from headers. Presigned URLs carry all SigV4 material in the query string and use a different canonical request, so presigned requests are currently treated as anonymous → private objects 403.
Most of the machinery already exists: verify_sigv4_signature takes a query_string and canonicalize_query_string sorts it; the proxy already plumbs signing_path/signing_query into RequestInfo (lib.rs:162-170).
Add to multistore (auth/sigv4.rs + auth/identity.rs):
- Detect presigned requests (
X-Amz-Algorithm=AWS4-HMAC-SHA256/X-Amz-Signaturein query) before theAnonymousearly-return. - Parse SigV4 from query (
parse_sigv4_presigned):X-Amz-Credential,X-Amz-SignedHeaders,X-Amz-Signature,X-Amz-Date,X-Amz-Expires,X-Amz-Security-Token. - Presigned canonical-request variant: canonical query = all params except
X-Amz-Signature; payload hash = literalUNSIGNED-PAYLOAD; date from theX-Amz-Datequery param. Reuse signing-key derivation +constant_time_eq. - Enforce expiry (
X-Amz-Date + X-Amz-Expires); effective lifetime =min(expires window, sealed STS token exp). - Resolve
X-Amz-Security-Tokenvia the existingcredential_resolver→ sameResolvedIdentity::Authenticated, same downstream authorize → presign-backend → stream path.
Ship as a multistore minor release.
Changes in this repo (minimal — mostly free after the bump)
Cargo.toml: bumpmultistore(+ siblings) to the release adding query-auth.src/lib.rs: verify the inboundhostheader matches what clients sign (SignedHeaders=host); add a test. Confirmadd_corsaccess-control-expose-headerscoverscontent-range/content-length/etag/accept-rangesfor browser range requests.src/lib.rs::log_analytics(optional follow-up):user_idis read from thex-source-user-idheader, which presigned viewer requests won't send — optionally source it from the verified STS principal. Bytes/account/product/key/geo still log correctly.tests/: presigned integration tests (see below).
Minting (separate repos — Source API + frontend)
The proxy is a pure verifier. Recommended: mint server-side in the Source API (authz authority; avoids shipping creds to the browser):
- Obtain STS temp creds via the existing
POST /.sts/assume-role-with-web-identity(audiencehttps://data.source.coop). - Presign a GET against
https://data.source.coop/{account}/{product}/{key}(@aws-sdk/s3-request-presignerwithforcePathStyle: true, or boto3generate_presigned_url), passing theSessionToken.X-Amz-Expires≤ rolemax_session_duration. - Return the flat URL; frontend drops it into the viewer iframe.
Phasing
- multistore: inbound presigned SigV4 verification + expiry + AWS presigned test vectors → minor release.
- this repo: dependency bump; host-canonicalization + presigned integration tests; CORS expose-headers; (optional) STS-principal
user_idin analytics. - Source API + frontend: "get viewer URL" mint endpoint, wired into the iframe viewers.
- private data: unblocked by the separate backend-credentials workstream — no further proxy change needed once non-public backends are readable.
Verification
- multistore unit: AWS SigV4 presigned test vectors; expired URL → denied.
- Integration (
wrangler dev+tests/): presign a public object → 200; tamper a query param → 403SignatureDoesNotMatch; range request → 206 + correctcontent-range; confirm an Analytics Engine event with non-zerobytes_sent(proves bytes still flow through the worker) and that a public-product presigned GET still triggers the location broadcast;aws s3 presign s3://.../key --endpoint-url …end-to-end.
Strategy drafted by Claude Code.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with multistore-0.4.0/src/auth/identity.rs and auth/sigv4.rs, then inspect src/lib.rs and the existing tests in tests/. Verify the dependency bump, host validation, and presigned integration behavior, including tampering, expiry, range responses, byte metering, and location broadcasts. Done means standard AWS SigV4 presigned GET/HEAD requests authenticate correctly while writes remain rejected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, rust
- Domain
- api, authentication, backend, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 38/100