Dstack-TEE / Dstack-TEE/private-ai-gateway
Bind Privatemode's active manifest to each inference receipt
- Dominant language
- Rust
- Stars
- 32
- Forks
- 8
- Avg merge
- 11h 9m
- Merged PRs (30d)
- 36
Description
## Problem
The Privatemode proxy supports dynamic manifest refresh, but v1.48 does not expose which manifest produced the inference secret used for a request.
The proxy's manifest history is insufficient for a security binding:
- dynamic mode writes a fetched manifest to the history before Coordinator attestation succeeds;
- the in-memory current manifest changes after Mesh CA verification but before secret exchange succeeds;
- `/v1/models` does not use or identify the active inference secret;
- the proxy CLI discards its `CurrentManifest` accessor;
- the request secret ID is not mapped to a manifest through the proxy API.
The gateway therefore reports the latest manifest-history digest as explicitly unbound observation metadata (`manifest_bound_to_active_secret: false`). It must leave manifest-specific GPU, OS, serving-software, and model-weight claims unknown.
## Required upstream contract
Add an atomic encryption-epoch record inside the Privatemode proxy:
```text
secret_id -> manifest_sha256, activated_at
```
Update it only after both Coordinator verification and secret exchange succeed. Do not update it when a manifest is merely fetched or when secret exchange fails.
For each successfully authenticated inference response, expose the exact epoch used by that attempt through local response headers or equivalent structured metadata:
```text
Privatemode-Secret-ID:
Privatemode-Manifest-SHA256: <64-hex>
```
The metadata must work for buffered and streaming responses and must identify the secret that actually encrypted/decrypted that attempt, including retries and rotations.
## Gateway follow-up
Once the proxy exposes this contract:
- bind `manifest_sha256` and `secret_id` into the signed receipt and attested session for the served request;
- match `(reviewed proxy image digest, active manifest digest)` against explicit reviewed profiles;
- emit manifest-specific `VerifierDerived` GPU, OS, serving-software, and model-weight claims only on an exact reviewed-profile match;
- continue to expose unknown manifests exactly, but leave those transitive claims `Unknown`;
- create a new content-addressed session when the active encryption epoch rotates;
- remove or demote the manifest-history observation once the request-bound value is available.
## Acceptance criteria
- A failed manifest fetch, Coordinator verification, or secret exchange cannot change the reported active epoch.
- A response cannot cite a manifest from a different secret or retry attempt.
- Rotation is covered for buffered and SSE responses.
- Unknown but valid dynamic manifests continue serving without receiving reviewed-profile claims.
- Tests demonstrate that a fetched-but-rejected manifest never appears as the request-bound manifest.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the Privatemode proxy's manifest refresh, Coordinator verification, secret exchange, and inference response paths, then follow the gateway's signed receipt and attested-session handling. Define the atomic secret-to-manifest epoch record and response metadata contract, and use the acceptance criteria to cover failed fetches, rejected manifests, retries, rotations, buffered responses, and SSE responses.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend-api-design, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100