NVIDIA / NVIDIA/nvcf

Add ModelExpress server configuration for self-managed clusters

Open
#1,345 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
218
Forks
72
Avg merge
1d 12h
Merged PRs (30d)
427

Description

Description

Add the cluster-admin values file that installs the upstream ModelExpress chart
on a self-managed NVCF compute plane, plus the CRD installation step.

Several upstream chart defaults do not work as shipped, so a default install is
not a usable starting point. Two of them stop the server outright. Each override
needs an inline reason, so a reviewer can tell an override apart from a
preference.

The server, its CRDs, and its RBAC are installed by an administrator rather than
shipped inside a function chart. The CRDs are a cluster-level installation and
the server is shared by every function that uses it.

Definition of Done

  • A values file overrides every upstream default that is unsafe or non-working,
    with each override explained inline.
  • The required metadata backend is set, since the server does not start without
    it.
  • The server image is pinned to the client version from the version tuple task,
    not the chart version.
  • RBAC is enabled for the chosen backend and stays namespaced, with no
    ClusterRole.
  • Model and registry credentials are referenced from secrets, never inlined.
  • helm template of the upstream chart with this file renders the intended
    image, service account, and RBAC.

Resources

Five upstream defaults were found to need overriding, all confirmed by
installing the chart rather than by reading it:

  • MX_METADATA_BACKEND is unset and required. The server does not start.
  • runAsNonRoot: true with no runAsUser, in both podSecurityContext and
    securityContext, cannot start the chart's own image, which runs as root. The
    failure is container has runAsNonRoot and image will run as root.
  • MODEL_EXPRESS_CACHE_DIRECTORY: /root is unwritable once the server runs as a
    non-root UID.
  • imagePullSecrets: nvcr-secret assumes a secret that may not exist. The server
    image is public, so the default should be empty.
  • persistence.size: 10Gi is sized for one small model.

Upstream chart: https://helm.ngc.nvidia.com/nvidia/ai-dynamo
Upstream CRDs: https://github.com/ai-dynamo/modelexpress/blob/v0.4.0/examples/crds.yaml

Part of #1113.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the upstream ModelExpress chart at helm.ngc.nvidia.com/nvidia/ai-dynamo and the CRD manifest examples/crds.yaml at v0.4.0. Create the cluster-admin values file from the listed unsafe defaults and render it with helm template. Done means the required backend, pinned client-version image, namespaced RBAC, secret references, and intended service account render successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes
Domain
devops, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.