nebari-dev / nebari-dev/llm-serving-pack

EPP Role RBAC: operator ClusterRole missing inference.networking.x-k8s.io grants (served models fail on Helm install)

Open
#121 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
3
Forks
3
Avg merge
16h 31m
Merged PRs (30d)
11

Description

Found while testing #117 live on an AWS Nebari cluster (Helm/ArgoCD install).

On a Helm install, every served LLMModel fails to reconcile:

reconciling EPP Role: roles "<model>-epp" is forbidden: user
"system:serviceaccount:nebari-llm-serving-system:nebari-llm-serving-operator"
is attempting to grant RBAC permissions not currently held:
  inference.networking.x-k8s.io: inferenceobjectives, inferencemodelrewrites, inferencepools, inferencepoolimports (get/list/watch)

The per-model EPP Role rendered by operator/internal/controller/reconcilers/inferencepool.go grants get/list/watch on those inference.networking.x-k8s.io resources, but the operator's own ClusterRole (charts/nebari-llm-serving/templates/operator-clusterrole.yaml) does not hold them. Kubernetes privilege-escalation prevention then blocks the operator from creating the Role, and the LLMModel reconcile aborts before it creates the model's SecurityPolicy and AIGatewayRoute too.

make deploy on kind runs the operator with a broader role, which is why the kind validation did not catch it.

Impact: no served model can come up on a Helm/ArgoCD install.

Fix: add the matching inference.networking.x-k8s.io get/list/watch rule to the operator ClusterRole. Done on the #117 branch (commit 0640fb2b); verified the operator can create the EPP Role after the change (it failed with the escalation error before).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with charts/nebari-llm-serving/templates/operator-clusterrole.yaml and compare the operator permissions with the EPP Role rendered by operator/internal/controller/reconcilers/inferencepool.go. Review commit 0640fb2b, then validate with a Helm/ArgoCD install; done means the operator creates the EPP Role and the served LLMModel reconciles successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, helm, kubernetes
Domain
authorization, infrastructure
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.