NVIDIA-NeMo / NVIDIA-NeMo/Safe-Synthesizer
feat(config): expose vLLM enforce_eager and compilation_config
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 38
- Forks
- 11
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 9
Description
Is your feature request related to a problem?
Safe Synthesizer exposes generation.attention_backend, but operators cannot configure vLLM's enforce_eager or compilation_config without patching VllmBackend.
The current parameter model ends with the attention-backend control (GenerateParameters). Engine initialization passes that setting to vLLM but leaves eager execution and compilation at vLLM's defaults (VllmBackend.initialize).
This limits runtime debugging and compatibility work. vLLM 0.24 accepts both controls on LLM: enforce_eager and compilation_config.
PR #363 removed an internally derived enforce_eager value after subprocess isolation resolved the Unsloth conflict. The requested configuration should restore operator control without coupling eager mode to a training implementation.
Describe the solution you'd like
Add typed generation settings that preserve current behavior by default:
generation:
enforce_eager: false
compilation_config: null
Expected types:
generation.enforce_eager: bool = falsegeneration.compilation_config: int | dict[str, Any] | None = null
Pass both values through to the vLLM LLM constructor. A mapping should remain an atomic configuration leaf so vLLM owns validation of its version-specific compilation fields.
Document an example that disables model-graph compilation:
generation:
enforce_eager: true
compilation_config:
mode: 0
Define the CLI behavior for the mapping field explicitly. YAML and SDK inputs must support the full mapping form.
Acceptance criteria
GenerateParametersexposes both fields with defaults matching current vLLM behavior.VllmBackend.initialize()forwards resolved values tovLLM.- Existing configurations produce the same engine arguments as before.
- Unit tests cover defaults and explicit overrides, including a mapping-valued
compilation_config. - Configuration reference and running guidance explain eager-mode performance cost and the accepted compilation-config forms.
- Resume-time generation overrides accept both settings through the existing
generationsection.
Describe alternatives you've considered
Users can maintain a source patch or construct vLLM outside Safe Synthesizer. Both approaches bypass the supported YAML, CLI, and SDK configuration flow and make saved-run generation harder to reproduce.
Additional context
This control does not remove every runtime compilation requirement. Triton compiles its CUDA host-driver shim on first GPU use, so enforce_eager: true does not replace the compiler dependency fixed by #682. Documentation should keep that boundary clear.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with GenerateParameters in src/nemo_safe_synthesizer/config/generate.py and VllmBackend.initialize in src/nemo_safe_synthesizer/generation/vllm_backend.py. Trace the existing generation configuration and resume-time overrides, then inspect the configuration reference and running guidance locations. Done means defaults preserve existing engine arguments, explicit boolean and integer or mapping values reach vLLM, and unit tests and documentation cover the accepted forms and eager-mode tradeoffs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend, documentation
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100