NVIDIA / NVIDIA/NeMo-Relay

[Enhancement]: support private caller-credential use by native execution plugins

Open
#1,108 1 comment 0 reactions 1 assignee View on GitHub

@bbednarski9 is already working on this.

Since Sep 17, 2026.

Improvement
Dominant language
Rust
Stars
173
Forks
73
Avg merge
7h 48m
Merged PRs (30d)
269

Description

Affected area

Plugins; Rust core runtime; CLI gateway; observability or exporters.

Problem or opportunity

Native execution plugins that make their own provider calls need a supported way to use a caller's provider credential without exposing it in observable request headers.

This limitation follows from the public source:

Consequently, a Switchyard route using forward_auth = true cannot obtain the original caller credential from the gateway-provided request. This is a source-level finding; no new binary or live-provider reproduction is claimed here.

Proposed enhancement

Add an explicit, request-scoped provider-call capability for native execution plugins. It should apply the caller's provider credential privately to an authorized upstream call, without adding the secret to general plugin/event headers.

A host-owned dispatch capability or opaque credential handle could provide this. Define which destinations and provider families may use it, including routing-model calls, answer calls, retries, and fallbacks.

Runtime contract and binding impact

Start with Rust/native plugins and document capability discovery and minimum host/SDK versions. Preserve the same ownership rules if other bindings expose the capability later.

Keep gateway invocation credentials separate from provider credentials. Plugins must not automatically gain permission to forward credentials to arbitrary destinations. Existing observable-header filtering must remain intact.

Alternatives considered
  • Deployment-owned api_key_env credentials support shared deployment authentication, but cannot preserve each caller's own provider account.
  • Standalone Switchyard supports caller forwarding, but does not provide the native Relay integration.
  • Integrations can reject unsupported forwarding configurations until a capability is available.
  • Restoring credentials to observable headers would weaken the existing confidentiality boundary.
Acceptance criteria
  • A real native-plugin test uses synthetic credentials and a local upstream for Chat Completions and Responses, buffered and streaming.
  • Authorized calls receive the correct caller credential and required companion headers; concurrent requests cannot mix credentials.
  • Credentials remain absent from observable headers, events, logs, metrics, and exported traces.
  • Unauthorized destinations, redirects, and incompatible provider families cannot receive source credentials.
  • Routing-model calls, retries, and fallback candidates obey the same explicit policy.
  • Gateway invocation tokens never become provider credentials.
  • Missing credentials or unsupported capability returns a clear error.
  • Existing deployment-owned authentication, keyless targets, ordinary forwarding, and cancellation behavior remain supported.
Related integration change

Switchyard's startup-rejection mitigation is proposed in https://github.com/NVIDIA-NeMo/Switchyard/pull/759. That change rejects unsupported configurations; it does not implement this capability.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.