NVIDIA-NeMo / NVIDIA-NeMo/Curator
Centralize vLLM setup utilities across inference stages
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 327
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 30
Description
Context
The Nemotron-Parse inference stage (nemo_curator/stages/interleaved/pdf/nemotron_parse/inference.py) contains vLLM setup code (engine args, sampling params, GPU configuration) that is useful for any inference stage using vLLM.
Raised in PR #1693 (VibhuJawa): "Do we have a common way across curator to setup vLLM? If not, we should make this a common place/util to start. Given this is probably the scale at which we have tested this the most GPUs, I don't think we should duplicate this logic."
Proposed work
- Extract vLLM engine initialization, sampling parameter construction, and GPU/TP configuration into a shared util (e.g.
nemo_curator/backends/vllm_utils.pyor similar) - Refactor
NemotronParseInferenceStageto use the shared util - Document as the canonical starting point for future vLLM-based inference stages
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in nemo_curator/stages/interleaved/pdf/nemotron_parse/inference.py and inspect how NemotronParseInferenceStage builds vLLM engine arguments, sampling parameters, and GPU or tensor-parallel configuration. Extract the reusable setup into a shared backend utility, refactor this stage to use it, and document that utility as the canonical starting point for future vLLM inference stages.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100