feat: SGLang support
@linnanwang is already working on this.
Since Jun 23, 2026.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 349
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 318
Description
Use cases, pain points, and background
Several RL training frameworks (e,g, VeRL, Slime) offer SGLang as a generation backend. If Gym doesn't support SGLang, users whose training loop runs on SGLang must either stand up a separate vLLM instance just for Gym, or manually proxy requests — adding latency and operational complexity. A first-class sglang_model server would let these frameworks plug into Gym directly.
Description:
Add a new responses_api_models/sglang_model/ model server that targets a remote SGLang OpenAI-compatible endpoint, analogous to the existing vllm_model
Design:
i think there are a few key requirements we would need to translate from vllm to sglang side:
- on policy corrections in vllm (just need to find the cleanest way to insert this logic for sglang) https://github.com/NVIDIA-NeMo/RL/blob/cd54f8f492d013f19f3bfa3af6ba5d3aab530e52/nemo_rl/models/generation/vllm/vllm_worker_async.py#L41
- getting the logprobs and token IDs for a chat completion request https://github.com/NVIDIA-NeMo/Gym/blob/18384bfc428045cc65751eccde7d3da7ae4db6a2/responses_api_models/vllm_model/app.py#L502
- applying the responses <> chat completions converter https://github.com/NVIDIA-NeMo/Gym/blob/18384bfc428045cc65751eccde7d3da7ae4db6a2/responses_api_models/vllm_model/app.py#L141
- max seq len hit error handling (might be specific to vllm) https://github.com/NVIDIA-NeMo/Gym/blob/18384bfc428045cc65751eccde7d3da7ae4db6a2/responses_api_models/vllm_model/app.py#L457
Out of scope:
What are some items that this issue could be mistaken to cover that this issue should explicitly NOT cover?
Acceptance Criteria:
- sglang_model server starts, passes health checks, and serves /v1/chat/completions and /v1/responses
- Responses ↔ ChatCompletions conversion works correctly (tool calls, reasoning, multi-turn)
- Token ID information (prompt_token_ids, generation_token_ids, generation_log_probs) is returned when return_token_id_information: true
- ng_collect_rollouts completes successfully against an SGLang endpoint using sglang_model
- Unit tests pass with ≥ 95% coverage
- Config and dogs updated
- End-to-end RL training run (e.g. GRPO) using sglang_model converges comparably to the same run using vllm_model on the same benchmark and model checkpoint.
- Reward curves and pass@1 metrics match within acceptable variance (< 1%) across identical ng_collect_rollouts runs against both backends.
- Token IDs returned by SGLang are consistent with the tokenizer — round-tripping through create_tokenize, generate, and the returned prompt_token_ids / generation_token_ids produces the same token sequences as vLLM, with no re-tokenization drift.
- Log-probs from SGLang match vLLM's to within numerical precision (fp16/bf16 tolerance) for the same prompt and greedy decoding, confirming the training signal is equivalent.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.