Pluggable model routing RFC

Summary

Create and review the RFC for pluggable model routing: the component that selects an approved model endpoint for model-bound traffic based on signals such as sensitivity, data residency, cost, latency, model capability, availability, tenant policy, or workload policy.

This is the real GitHub-mirrored child issue for the Privacy Guard roadmap item. Model routing was part of the original Privacy Guard proposal, but it is a distinct design surface from sandbox egress middleware and should be reviewed independently.

Why this is separate

• Sandbox egress middleware produces metadata and findings, such as request modality, sensitivity labels, local-model requirements, and mitigation status.
• Model routing is the intended consumer of that metadata for model-bound traffic.
• Routing decisions and routing policy language are a larger product surface than privacy enforcement and should not block the v1 sandbox egress middleware RFC.

Scope

• Define a pluggable model-router contract with a simple default implementation.
• Define how model routing consumes middleware findings and request metadata without requiring another scan.
• Define the routing position in the request pipeline, including its relationship to egress policy and credential injection.
• Define how operators express and validate residency, sensitivity, cost, capability, availability, tenant, and workload routing rules.
• Define the minimum metadata contract needed for the sandbox egress middleware RFC to compose with future routing work.

Out of Scope

• Privacy detection, transformation, and egress middleware interface design, which are tracked in https://github.com/NVIDIA/OpenShell/issues/1733.
• Implementing every advanced routing strategy in the first version.
• Changing non-model-bound egress behavior beyond the metadata needed for model routing.

Grounding Notes

• crates/openshell-router/ currently acts as a protocol selector. It chooses the first route whose protocol matches, injects the API key, and rewrites the model field.
• The current router is not privacy-aware and has no request-metadata field or context.
• There is no current mechanism to thread middleware-produced metadata into a routing decision.

Open Questions

• Does v1 need a router implementation, or only a stable metadata contract that a future router can consume?
• Where exactly does routing sit relative to credential injection in the real supervisor relay path?
• How should routing policy be expressed and validated?

Related

• Parent roadmap issue: https://github.com/NVIDIA/OpenShell/issues/1043
• Sandbox egress middleware RFC: https://github.com/NVIDIA/OpenShell/issues/1733
• Earlier model-routing design notes were tracked in a non-GitHub-mirrored Linear issue and should not be used for PR linking.

Start with crates/openshell-router to understand the current protocol selection, API-key injection, and model rewriting, then inspect the supervisor relay path and issue 1733 for the middleware context. The RFC is done when the router contract, metadata flow, pipeline position, policy validation approach, and minimum composition metadata are defined and reviewed.