Add ModelExpress server configuration for self-managed clusters
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Description
Add the cluster-admin values file that installs the upstream ModelExpress chart
on a self-managed NVCF compute plane, plus the CRD installation step.
Several upstream chart defaults do not work as shipped, so a default install is
not a usable starting point. Two of them stop the server outright. Each override
needs an inline reason, so a reviewer can tell an override apart from a
preference.
The server, its CRDs, and its RBAC are installed by an administrator rather than
shipped inside a function chart. The CRDs are a cluster-level installation and
the server is shared by every function that uses it.
Definition of Done
- A values file overrides every upstream default that is unsafe or non-working,
with each override explained inline. - The required metadata backend is set, since the server does not start without
it. - The server image is pinned to the client version from the version tuple task,
not the chart version. - RBAC is enabled for the chosen backend and stays namespaced, with no
ClusterRole. - Model and registry credentials are referenced from secrets, never inlined.
helm templateof the upstream chart with this file renders the intended
image, service account, and RBAC.
Resources
Five upstream defaults were found to need overriding, all confirmed by
installing the chart rather than by reading it:
MX_METADATA_BACKENDis unset and required. The server does not start.runAsNonRoot: truewith norunAsUser, in bothpodSecurityContextand
securityContext, cannot start the chart's own image, which runs as root. The
failure iscontainer has runAsNonRoot and image will run as root.MODEL_EXPRESS_CACHE_DIRECTORY: /rootis unwritable once the server runs as a
non-root UID.imagePullSecrets: nvcr-secretassumes a secret that may not exist. The server
image is public, so the default should be empty.persistence.size: 10Giis sized for one small model.
Upstream chart: https://helm.ngc.nvidia.com/nvidia/ai-dynamo
Upstream CRDs: https://github.com/ai-dynamo/modelexpress/blob/v0.4.0/examples/crds.yaml
Part of #1113.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the upstream ModelExpress chart at helm.ngc.nvidia.com/nvidia/ai-dynamo and the CRD manifest examples/crds.yaml at v0.4.0. Create the cluster-admin values file from the listed unsafe defaults and render it with helm template. Done means the required backend, pinned client-version image, namespaced RBAC, secret references, and intended service account render successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100