vllm-project / vllm-project/production-stack
feature: Terraform Quickstart Tutorials for Underlying Infrastructure
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 503
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 8
Description
Describe the feature
While the tutorials to build production stack across different clouds using Bash scripts are a great start, many professional teams will make sure of infrastructure tooling such as Terraform to manage Kubernetes clusters and other required resources. It would be good to add some base Terraform code to deploy the required K8s cluster, required NVIDIA plugin, and vLLM Production Stack to give users a way to get started that includes long-term management capabilities for clusters that are moving beyond a POC stage.
I plan to create a tutorial, starting with AWS EKS as the underlying K8s cluster, to help address this need. I already have a quickstart repo that uses Ollama and Open WebUI in a similar way, and will create a new tutorial within the Production Stack repo based on this: https://github.com/0xThresh/self-hosted-genai
Please let me know if there is any specific functionality you'd like to see as part of this tutorial!
Why do you need this feature?
No response
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the existing Bash-based production-stack tutorials and the linked self-hosted-genai quickstart repo before defining the tutorial structure. The work should cover AWS EKS, the required NVIDIA plugin, and the vLLM Production Stack through Terraform, with a working quickstart that documents how users deploy and manage the cluster beyond a proof of concept.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes, terraform
- Domain
- cloud, documentation, infrastructure
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100