NVIDIA-NeMo / NVIDIA-NeMo/Automodel
Support for vLLM in 26.04
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 963
- Forks
- 318
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 143
Description
Is your feature request related to a problem? Please describe.
Add vLLM in our docker container to streamline the train-inference workflow.
Describe the solution you'd like
We want to include tests in our Gha CI and the software on the docker container.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
I'm adding @thomasdhc for the automation, but will update assignees on code side.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Docker container definition and GitHub Actions CI workflows used by Automodel. Determine how vLLM should be added to the container and which CI tests should cover the train-inference workflow. Done means the container includes vLLM and the relevant tests run successfully in GitHub Actions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, github-actions, python
- Domain
- ci-cd, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100