Implement Keycloak admin lookup for API key audit

Today key-manager/cmd/main.go:makeUserinfoLookup is a stub that returns an error, so the audit loop never revokes keys. External API keys remain valid after a user is removed from a model's Keycloak groups. Users lose access to the internal JWT endpoint immediately (the gateway checks spec.access.groups on every request - #47), but there is no time-bounded revocation path for external API keys other than deleting the LLMModel CR.

Why the current env var is misleading

LLM_OIDC_USERINFO_URL is the wrong primitive. OIDC /userinfo returns claims for the token bearer; the auditor has only a username string, not a user token. Calling /userinfo with the key manager's own service-account token returns the service account's claims, not the target user's.

The fix requires looking up users by username, which is an admin-only operation.

Suggested implementation

A new key-manager/internal/keycloak package that:

1. Authenticates to Keycloak using OAuth2 client credentials (POST /realms/{realm}/protocol/openid-connect/token, grant_type=client_credentials).
2. Calls the admin API: GET /admin/realms/{realm}/users?exact=true&username={username} to resolve the user, then GET /admin/realms/{realm}/users/{id}/groups for the group list.
3. Caches tokens until expiry and refreshes them transparently.
4. Returns the list of group names (with the leading / Keycloak uses in group paths) to the auditor, which already does case-sensitive string matching against spec.access.groups.

Configuration (env vars on the key-manager pod):

• LLM_KEYCLOAK_BASE_URL - e.g. https://keycloak.example.com
• LLM_KEYCLOAK_REALM - e.g. nebari
• LLM_KEYCLOAK_CLIENT_ID - the service-account client's client ID
• LLM_KEYCLOAK_CLIENT_SECRET_FILE - path to a file containing the client secret (mount the Secret into the pod rather than exposing the value via env)

Deprecate LLM_OIDC_USERINFO_URL and keyManager.oidcUserinfoURL in the chart; keep them for one release with a warning log on startup if they're set.

Setting up the Keycloak client

This is pre-req work the pack operator has to do manually:

1. Create a Keycloak client in the target realm (e.g. nebari-llm-key-audit).
2. Enable "Service Accounts Enabled"; disable direct access grants / standard flow.
3. Under the client's "Service Account Roles" tab, assign the view-users role from the realm-management client.
4. Generate a client secret and store it as a Kubernetes Secret in the operator namespace.

Alternative: automate via NebariApp or a new CRD

Follow-up possibility once the manual path is in place: let the nebari-operator provision the admin-role service-account client alongside the UI client (today provisionClient: true only yields a public client with end-user roles). Tracked separately when it comes up.

Definition of done

• Auditor removes external API keys within auditInterval of a user losing group access, verified end-to-end against a real Keycloak realm.
• Chart docs (docs/design.md §"API key audit", values.yaml comment) describe the manual setup steps or point at an operator-automated path.
• Metrics or log lines allow operators to see how many keys have been revoked, how many lookups failed, and distinguish "fail-safe skip" from "user genuinely has no groups."

References

• Keycloak Admin REST API: search users
• Keycloak Admin REST API: get user groups
• Related: #47 (gateway enforcement on internal path), existing auditor framework at key-manager/internal/audit/auditor.go.

Start with key-manager/cmd/main.go:makeUserinfoLookup and key-manager/internal/audit/auditor.go, then inspect the chart configuration, docs/design.md §"API key audit", and values.yaml. Trace the existing audit loop before designing the Keycloak admin client, token caching, deprecation warnings, and observability. Done means group loss revokes keys within auditInterval in a real Keycloak realm and the manual setup is documented.