source-cooperative / source-cooperative/data.source.coop

Support standard SigV4 presigned URLs (inbound query-string auth)

Open
#136 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
24
Forks
6
Avg merge
1h 32m
Merged PRs (30d)
1

Description

Summary

Support standard AWS SigV4 presigned URLs against data.source.coop so the frontend can hand a flat, self-authorizing GET URL to data-viewer iframes (COG/GeoTIFF, PMTiles, GeoParquet, …) — including for private/restricted products — without embedding the user's session credentials in the iframe.

Presigned URLs point back at the proxy (not the raw S3/Azure/GCS bucket), so the worker still performs the backend fetch and streams the bytes. This preserves per-download byte metering and the live location broadcast (a hard requirement) — only identity resolution changes, not the data path.

Decisions (scoping)

  • Standard SigV4 presigned URLs, verified against the existing STS stack — not a bespoke token scheme. Stays interoperable with aws CLI / GDAL /vsis3/ / SDKs and reuses the STS investment.
  • Backend credentials for private data are a separate workstream. This work assumes the proxy can already read non-public backends (today registry.rs:188 hardcodes skip_signature=true / anonymous and DataConnectionDetails carries no credential fields). This issue covers only the presigned-URL minting + verification layer.
  • GET/HEAD only (proxy is read-only; writes → 405). No presigned uploads.

The crux: inbound presigned (query-string) SigV4 verification

The core gap lives in the multistore crate (co-maintained). multistore-0.4.0/src/auth/identity.rs::resolve_identity returns Anonymous whenever there's no Authorization header, and reads the session token / payload hash from headers. Presigned URLs carry all SigV4 material in the query string and use a different canonical request, so presigned requests are currently treated as anonymous → private objects 403.

Most of the machinery already exists: verify_sigv4_signature takes a query_string and canonicalize_query_string sorts it; the proxy already plumbs signing_path/signing_query into RequestInfo (lib.rs:162-170).

Add to multistore (auth/sigv4.rs + auth/identity.rs):

  1. Detect presigned requests (X-Amz-Algorithm=AWS4-HMAC-SHA256 / X-Amz-Signature in query) before the Anonymous early-return.
  2. Parse SigV4 from query (parse_sigv4_presigned): X-Amz-Credential, X-Amz-SignedHeaders, X-Amz-Signature, X-Amz-Date, X-Amz-Expires, X-Amz-Security-Token.
  3. Presigned canonical-request variant: canonical query = all params except X-Amz-Signature; payload hash = literal UNSIGNED-PAYLOAD; date from the X-Amz-Date query param. Reuse signing-key derivation + constant_time_eq.
  4. Enforce expiry (X-Amz-Date + X-Amz-Expires); effective lifetime = min(expires window, sealed STS token exp).
  5. Resolve X-Amz-Security-Token via the existing credential_resolver → same ResolvedIdentity::Authenticated, same downstream authorize → presign-backend → stream path.

Ship as a multistore minor release.

Changes in this repo (minimal — mostly free after the bump)

  • Cargo.toml: bump multistore (+ siblings) to the release adding query-auth.
  • src/lib.rs: verify the inbound host header matches what clients sign (SignedHeaders=host); add a test. Confirm add_cors access-control-expose-headers covers content-range/content-length/etag/accept-ranges for browser range requests.
  • src/lib.rs::log_analytics (optional follow-up): user_id is read from the x-source-user-id header, which presigned viewer requests won't send — optionally source it from the verified STS principal. Bytes/account/product/key/geo still log correctly.
  • tests/: presigned integration tests (see below).

Minting (separate repos — Source API + frontend)

The proxy is a pure verifier. Recommended: mint server-side in the Source API (authz authority; avoids shipping creds to the browser):

  1. Obtain STS temp creds via the existing POST /.sts/assume-role-with-web-identity (audience https://data.source.coop).
  2. Presign a GET against https://data.source.coop/{account}/{product}/{key} (@aws-sdk/s3-request-presigner with forcePathStyle: true, or boto3 generate_presigned_url), passing the SessionToken. X-Amz-Expires ≤ role max_session_duration.
  3. Return the flat URL; frontend drops it into the viewer iframe.

Phasing

  1. multistore: inbound presigned SigV4 verification + expiry + AWS presigned test vectors → minor release.
  2. this repo: dependency bump; host-canonicalization + presigned integration tests; CORS expose-headers; (optional) STS-principal user_id in analytics.
  3. Source API + frontend: "get viewer URL" mint endpoint, wired into the iframe viewers.
  4. private data: unblocked by the separate backend-credentials workstream — no further proxy change needed once non-public backends are readable.

Verification

  • multistore unit: AWS SigV4 presigned test vectors; expired URL → denied.
  • Integration (wrangler dev + tests/): presign a public object → 200; tamper a query param → 403 SignatureDoesNotMatch; range request → 206 + correct content-range; confirm an Analytics Engine event with non-zero bytes_sent (proves bytes still flow through the worker) and that a public-product presigned GET still triggers the location broadcast; aws s3 presign s3://.../key --endpoint-url … end-to-end.

Strategy drafted by Claude Code.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with multistore-0.4.0/src/auth/identity.rs and auth/sigv4.rs, then inspect src/lib.rs and the existing tests in tests/. Verify the dependency bump, host validation, and presigned integration behavior, including tampering, expiry, range responses, byte metering, and location broadcasts. Done means standard AWS SigV4 presigned GET/HEAD requests authenticate correctly while writes remain rejected.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, rust
Domain
api, authentication, backend, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.