Dstack-TEE / Dstack-TEE/private-ai-gateway
Middleware plumbing: per-request upstream auth so a middleware can forward caller credentials (BYOK)
- Dominant language
- Rust
- Stars
- 32
- Forks
- 8
- Avg merge
- 11h 9m
- Merged PRs (30d)
- 36
Description
## Problem
The gateway authenticates to an upstream with a single, statically-configured per-upstream `bearer_token` (`OpenAICompatibleBackend`). The caller's inbound `Authorization` is consumed only as a hashed receipt-owner tag (`ReceiptOwner::from_bearer`) and is **never forwarded upstream**. `UpstreamRequest.headers` is always built empty at the forward sites.
This is correct for a gateway fronting one account, but it blocks **multi-tenant BYOK (bring-your-own-key)**: a fronting service can't let each end user supply their own upstream provider API key (e.g. a per-user NEAR AI key) and have the gateway relay it while still doing the attested-confidential-inference work the gateway exists for.
## Proposal
An opt-in per-upstream `auth_passthrough` mode: when enabled, the gateway forwards the caller's own `Authorization` to the upstream instead of a static token, and never uses a static token for that upstream.
Design points worth getting right:
- Mutually exclusive with a configured `bearer_token` (reject at config validation).
- Fail-closed: absent caller credential ⇒ no `Authorization` header (a visible upstream rejection), never a silent fallback.
- The raw credential must never reach logs, receipts, or the request store (a redacting newtype, end-to-end).
- Only providers served by the OpenAI-compatible forwarder honor it; reject it for providers whose backend ignores it (e.g. Chutes).
- Unsupported in middleware mode (the request store keeps only a hashed requester) — fail closed at startup.
Attestation, verification, channel binding, and receipts are unchanged.
## Use case
bitrouter routes confidential inference through the gateway and wants each user's own provider key relayed per-request, with the gateway still attesting the upstream TEE and signing receipts.
PR follows.
Contributor guide
No contributing guide indexed for this repository
Research direction
Trace the OpenAICompatibleBackend configuration and UpstreamRequest.headers at the forward sites, then inspect ReceiptOwner::from_bearer and the request store. Verify configuration rejects conflicting or unsupported modes, missing credentials fail closed, and raw credentials do not enter logs, receipts, or stored requests; confirm Chutes and middleware mode are rejected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- authentication, backend, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100