Contribution offer: Terraform (IaC) deploy example - Dynamo on Nebius managed K8s
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 8.1k
- Forks
- 1.6k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 446
Description
Hi team, following a conversation with an NVIDIA contact who suggested opening an issue here, I'd like to offer a cloud + Terraform deployment example for Dynamo.
Repo: https://github.com/CloudThrill/nvidia-dynamo-stack-terraform (Nebius MK8s today, structured for per-cloud subfolders)
How it differs from the existing k8s examples. The current examples start from "you have a cluster with the operator installed" and walk through manifests. This one starts from an empty cloud project: terraform apply provisions the VPC, managed Kubernetes, GPU node group, ingress + cert-manager, kube-prometheus-stack, the dynamo-platform chart and the DGD, one command rather than a copy-paste sequence. As far as I can tell there's no IaC example in the repo today.
What's in it: all verified on a live 1.3.0 cluster:
- Multi-model behind one shared Frontend on a single GPU, including the per-worker
--endpoint dyn://$DYN_NAMESPACE-$DYN_NAMESPACE_WORKER_SUFFIX.<name>.generateneeded so the second model isn't rejected on the defaultbackendcomponent - Operator-managed ingress + Let's Encrypt TLS (the operator references a TLS secret it doesn't create, the example ships the cert-manager
Certificatethat fills it) - Prometheus/Grafana with the shipped Dynamo + operator dashboards, including the monitor selector relaxation needed for them to be scraped
- Optional KV-aware routing example (
router-mode=kv+ worker KV events): verified: 20/20 shared-prefix requests routed to the cache-warm replica - Measured deployment timings and a troubleshooting section built from real failures
- Real HTTPS URL endpoints no IP no port forwarding during the demos
Pinned to Dynamo 1.3.0 / vllm-runtime:1.3.0.
Placement : your call entirely; you know the repo layout better than I do. My instinct would be a self-contained terraform/nebius/ entry alongside the existing k8s examples: same shape as what's already there, just with the cluster provisioning included, so it stays isolated from the rest of the tree.
If carrying IaC in-tree isn't something the project wants, a short docs page linking out works too, though it'd be easier to find sitting with the other examples.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the linked CloudThrill/nvidia-dynamo-stack-terraform repository and the existing Kubernetes examples in this repository. Confirm whether the contribution should be an in-tree terraform/nebius/ example or a documentation link, then verify that the deployment, monitoring, TLS, and troubleshooting flow is reproducible against the pinned Dynamo 1.3.0 setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, prometheus, terraform
- Domain
- cloud, devops, infrastructure, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100