vllm-project / vllm-project/production-stack

feature: Terraform Quickstart Tutorials for Underlying Infrastructure

Open
#167 2 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request
Dominant language
Python
Stars
2.6k
Forks
503
Avg merge
4d 17h
Merged PRs (30d)
8

Description

Describe the feature

While the tutorials to build production stack across different clouds using Bash scripts are a great start, many professional teams will make sure of infrastructure tooling such as Terraform to manage Kubernetes clusters and other required resources. It would be good to add some base Terraform code to deploy the required K8s cluster, required NVIDIA plugin, and vLLM Production Stack to give users a way to get started that includes long-term management capabilities for clusters that are moving beyond a POC stage.

I plan to create a tutorial, starting with AWS EKS as the underlying K8s cluster, to help address this need. I already have a quickstart repo that uses Ollama and Open WebUI in a similar way, and will create a new tutorial within the Production Stack repo based on this: https://github.com/0xThresh/self-hosted-genai

Please let me know if there is any specific functionality you'd like to see as part of this tutorial!

Why do you need this feature?

No response

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the existing Bash-based production-stack tutorials and the linked self-hosted-genai quickstart repo before defining the tutorial structure. The work should cover AWS EKS, the required NVIDIA plugin, and the vLLM Production Stack through Terraform, with a working quickstart that documents how users deploy and manage the cluster beyond a proof of concept.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes, terraform
Domain
cloud, documentation, infrastructure
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.