Delegated worker token authentication for self-hosted NVCF

Is this related to a problem? Please describe.

Workers on self-hosted NVCF clusters authenticate using a bootstrap/refresh-token flow that requires persisting credentials to disk. Self-hosted nodes have no consistently present, cross-node persistence layer, so a node restart can leave a worker unable to re-authenticate until a new bootstrap credential is issued.

Describe the solution you'd like

Replace the bootstrap/refresh-token flow with short-lived Kubernetes Projected ServiceAccount Tokens (PSATs) issued per worker pod and verified by ICMS.

ICMS (merged): Extend the instance status update endpoint with a workerAuth payload so NVCA can register per-worker identities alongside each status update. Store those identities in Cassandra keyed by (cluster_id, instance_id). Expose POST /v1/workers/tokens/introspect (RFC 7662) so other services can verify a worker-presented JWT and receive an active/inactive verdict with resolved instance and worker context.

NVCA: Provision a worker ServiceAccount and projected SAT volume for each container function pod, injecting the token path and identity source as environment variables. Create an empty Role and RoleBinding for each worker SA to make the deny-by-default permission boundary explicit. Register the worker identity with ICMS on each status update. Clean up all worker identity objects on pod termination. Block Helm chart workloads from using nvcf-worker-* ServiceAccount names via a validating webhook.

Worker clients: Read the mounted projected SAT as the primary token source. Fall back to the existing bootstrap token when the mounted path is absent (managed NVCF or flag off).

NVCF API and NVCT API: When a worker presents a JWT that fails legacy decryption, forward it to the ICMS introspection endpoint. Authorize the request if the verdict is active. Gate on a per-service feature flag.

Deploy manifests: Helm chart and config updates to enable the end-to-end flow on self-hosted clusters.

Describe alternatives you've considered

• Keeping the bootstrap/refresh-token flow and adding node-level secret persistence (e.g., a host-path volume or a dedicated secret store). Rejected because it adds operational complexity and a persistent attack surface on every worker node.

• Issuing worker tokens from a central NVCF service rather than delegating verification to ICMS. Rejected because it would require the worker to make an outbound call to NVCF before every authenticated request, adding latency and a new dependency on NVCF reachability from the cluster.

Additional context

The OIDC/JWKS cluster-identity foundation (cluster registration, JWKS storage and rotation, audience-based cluster resolution) is already in place from the Multi-Cluster Registration feature. This work extends it from cluster identity to per-worker identity.

The issue names no files or tests; start with the ICMS instance status update endpoint and the proposed worker token introspection endpoint, then trace the NVCA, worker client, API, and Helm deployment entry points. Done means projected worker tokens, identity registration and cleanup, introspection-based authorization, legacy fallback, feature flags, and self-hosted deployment configuration work end to end.