NVIDIA-NeMo / NVIDIA-NeMo/Safe-Synthesizer

feat(config): expose vLLM enforce_eager and compilation_config

Open
#698 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature
Dominant language
Python
Stars
38
Forks
11
Avg merge
4d 17h
Merged PRs (30d)
9

Description

Is your feature request related to a problem?

Safe Synthesizer exposes generation.attention_backend, but operators cannot configure vLLM's enforce_eager or compilation_config without patching VllmBackend.

The current parameter model ends with the attention-backend control (GenerateParameters). Engine initialization passes that setting to vLLM but leaves eager execution and compilation at vLLM's defaults (VllmBackend.initialize).

This limits runtime debugging and compatibility work. vLLM 0.24 accepts both controls on LLM: enforce_eager and compilation_config.

PR #363 removed an internally derived enforce_eager value after subprocess isolation resolved the Unsloth conflict. The requested configuration should restore operator control without coupling eager mode to a training implementation.

Describe the solution you'd like

Add typed generation settings that preserve current behavior by default:

generation:
  enforce_eager: false
  compilation_config: null

Expected types:

  • generation.enforce_eager: bool = false
  • generation.compilation_config: int | dict[str, Any] | None = null

Pass both values through to the vLLM LLM constructor. A mapping should remain an atomic configuration leaf so vLLM owns validation of its version-specific compilation fields.

Document an example that disables model-graph compilation:

generation:
  enforce_eager: true
  compilation_config:
    mode: 0

Define the CLI behavior for the mapping field explicitly. YAML and SDK inputs must support the full mapping form.

Acceptance criteria
  • GenerateParameters exposes both fields with defaults matching current vLLM behavior.
  • VllmBackend.initialize() forwards resolved values to vLLM.
  • Existing configurations produce the same engine arguments as before.
  • Unit tests cover defaults and explicit overrides, including a mapping-valued compilation_config.
  • Configuration reference and running guidance explain eager-mode performance cost and the accepted compilation-config forms.
  • Resume-time generation overrides accept both settings through the existing generation section.

Describe alternatives you've considered

Users can maintain a source patch or construct vLLM outside Safe Synthesizer. Both approaches bypass the supported YAML, CLI, and SDK configuration flow and make saved-run generation harder to reproduce.

Additional context

This control does not remove every runtime compilation requirement. Triton compiles its CUDA host-driver shim on first GPU use, so enforce_eager: true does not replace the compiler dependency fixed by #682. Documentation should keep that boundary clear.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with GenerateParameters in src/nemo_safe_synthesizer/config/generate.py and VllmBackend.initialize in src/nemo_safe_synthesizer/generation/vllm_backend.py. Trace the existing generation configuration and resume-time overrides, then inspect the configuration reference and running guidance locations. Done means defaults preserve existing engine arguments, explicit boolean and integer or mapping values reach vLLM, and unit tests and documentation cover the accepted forms and eager-mode tradeoffs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.