Wire Dynamo workers to ModelExpress in the sample chart
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Description
Add the NVCF function chart: a DynamoGraphDeployment whose vLLM workers load
weights through ModelExpress, with a switch to run the identical topology on the
native loader for comparison.
Provider-neutral by default. Fabric-specific settings live in a separate override
file, so the chart carries no cloud-specific values.
Definition of Done
- Worker replica count is the scale-out dimension, and the chart renders
correctly at 2, 4, and 10 replicas. - vLLM load format names the ModelExpress loader. Environment variables alone
configure a loader vLLM never calls, which looks like success while
ModelExpress does nothing. - The client server address is set under both the current and the older variable
name, since the 0.4.0 client reads both. - The vLLM plugin is named explicitly, because the delivered vLLM is below 0.23.
- The NIXL backend is configurable, and the fabric-specific choice is documented
with the reason. - Disabling ModelExpress emits no ModelExpress environment at all, so the
baseline is a clean comparison. - No plaintext secret renders. The model token is optional, since workers reading
from a peer do not need one. helm lintpasses andhelm templateis deterministic.- An override sample carries the EFA settings and the untested InfiniBand and
RoCE settings, clearly labelled.
Resources
Backend choice, which is the part most likely to be got wrong. The client accepts
UCX or LIBFABRIC and defaults to UCX. AWS EFA needs LIBFABRIC together
with the vllm-runtime:1.2.1-efa-amd64 image, because the stock image ships no
libfabric.so.1 and the plugin cannot load without it. This is the combination
that was validated end to end.
The UCX default carries a known defect. On this client version the source
worker segfaulted while serving a transfer, in nixlUcxSharedThread::run(), and
it reproduced over plain TCP, so it is not fabric-specific. That is disclosed in
the chart values, the override sample, and the guide, because InfiniBand and RoCE
users take the same path and neither fabric was available to test.
Three deployment failures worth encoding, all hit in practice:
vpc.amazonaws.com/efamust be declared underlimits, notrequests.
Kubernetes rejects a Pod that requests an extended resource without a matching
limit, and the Dynamo operator passes the block through verbatim. The result is
ERR_CREATE_PODand pods that never appear, which reads as a scheduling
problem rather than a spec problem.- EFA caps workers at one per node. G-family instances expose a single EFA
interface and the device plugin advertises one unit per node, so replicas must
scale with nodes rather than with GPUs per node. - The locked-memory limit cannot be fixed from the chart. It is inherited from
the node's container runtime, andCAP_IPC_LOCKdoes not help because the
image runs as UID 1000 and a non-root process has an empty effective capability
set.
Existing structure reference: examples/function-samples/helmchart-samples/dynamo-operator-sample/
Override convention: examples/function-samples/helmchart-samples/multi-node-helm-function-test/override.yaml.sample
Part of #1113.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing examples/function-samples/helmchart-samples/dynamo-operator-sample/ with examples/function-samples/helmchart-samples/multi-node-helm-function-test/override.yaml.sample. Trace the chart values and templates for replica scaling, loader configuration, secrets, NIXL, and fabric overrides, then run helm lint and deterministic helm template renders. Done means the required topologies, clean disabled baseline, safe secret handling, documented overrides, and validation checks all pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100