Dstack-TEE / Dstack-TEE/private-ai-gateway

Middleware plumbing: per-request upstream auth so a middleware can forward caller credentials (BYOK)

Open
#11 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32
Forks
8
Avg merge
11h 9m
Merged PRs (30d)
36

Description

## Problem

The gateway authenticates to an upstream with a single, statically-configured per-upstream `bearer_token` (`OpenAICompatibleBackend`). The caller's inbound `Authorization` is consumed only as a hashed receipt-owner tag (`ReceiptOwner::from_bearer`) and is **never forwarded upstream**. `UpstreamRequest.headers` is always built empty at the forward sites.

This is correct for a gateway fronting one account, but it blocks **multi-tenant BYOK (bring-your-own-key)**: a fronting service can't let each end user supply their own upstream provider API key (e.g. a per-user NEAR AI key) and have the gateway relay it while still doing the attested-confidential-inference work the gateway exists for.

## Proposal

An opt-in per-upstream `auth_passthrough` mode: when enabled, the gateway forwards the caller's own `Authorization` to the upstream instead of a static token, and never uses a static token for that upstream.

Design points worth getting right:
- Mutually exclusive with a configured `bearer_token` (reject at config validation).
- Fail-closed: absent caller credential ⇒ no `Authorization` header (a visible upstream rejection), never a silent fallback.
- The raw credential must never reach logs, receipts, or the request store (a redacting newtype, end-to-end).
- Only providers served by the OpenAI-compatible forwarder honor it; reject it for providers whose backend ignores it (e.g. Chutes).
- Unsupported in middleware mode (the request store keeps only a hashed requester) — fail closed at startup.

Attestation, verification, channel binding, and receipts are unchanged.

## Use case

bitrouter routes confidential inference through the gateway and wants each user's own provider key relayed per-request, with the gateway still attesting the upstream TEE and signing receipts.

PR follows.

Contributor guide

No contributing guide indexed for this repository

Research direction

Trace the OpenAICompatibleBackend configuration and UpstreamRequest.headers at the forward sites, then inspect ReceiptOwner::from_bearer and the request store. Verify configuration rejects conflicting or unsupported modes, missing credentials fail closed, and raw credentials do not enter logs, receipts, or stored requests; confirm Chutes and middleware mode are rejected.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
authentication, backend, security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.