NVIDIA-NeMo / NVIDIA-NeMo/RL

Generation backends disagree on how to report "no OpenAI server": [None] vs []

Open
#3,977 0 comments 0 reactions 0 assignees View on GitHub
bug Feature
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

## Summary

`dp_openai_server_base_urls` is meant to hold one OpenAI-compatible base URL per data-parallel rank. When a backend has no HTTP server to report, the four backends disagree about what to put there: Megatron returns an empty list, the other three return a list of `None`s.

Megatron filters the `None`s out and can then use a plain truth test:

```python
# nemo_rl/models/generation/megatron/megatron_generation.py
if (
not self.dp_openai_server_base_urls # L368 — plain truth test works
and self.cfg["mcore_generation_config"]["expose_http_server"]
):
...
self.dp_openai_server_base_urls = [
url for url in ray.get(url_futures) if url is not None # L374-376
]
```
[megatron_generation.py:368](https://github.com/NVIDIA-NeMo/RL/blob/e9a855ff4bc0a41967e9491c41de093527e123b6/nemo_rl/models/generation/megatron/megatron_generation.py#L368),
[:374-376](https://github.com/NVIDIA-NeMo/RL/blob/e9a855ff4bc0a41967e9491c41de093527e123b6/nemo_rl/models/generation/megatron/megatron_generation.py#L374-L376)

The other three keep a placeholder entry instead:

| backend | what it returns | where |
|---|---|---|
| vLLM (sync engine) | `[None]` | [vllm_generation.py:529](https://github.com/NVIDIA-NeMo/RL/blob/e9a855ff4bc0a41967e9491c41de093527e123b6/nemo_rl/models/generation/vllm/vllm_generation.py#L529) |
| vLLM (deferred load, sync engine) | `[None]` | [vllm_generation.py:548](https://github.com/NVIDIA-NeMo/RL/blob/e9a855ff4bc0a41967e9491c41de093527e123b6/nemo_rl/models/generation/vllm/vllm_generation.py#L548) |
| TRT-LLM (no HTTP server) | `[None] * dp_size` | [trtllm_generation.py:266-267](https://github.com/NVIDIA-NeMo/RL/blob/e9a855ff4bc0a41967e9491c41de093527e123b6/nemo_rl/models/generation/trtllm/trtllm_generation.py#L266-L267) |
| Dynamo (no token wrapper) | `[None]` | [dynamo_generation.py:220](https://github.com/NVIDIA-NeMo/RL/blob/e9a855ff4bc0a41967e9491c41de093527e123b6/nemo_rl/models/generation/dynamo/dynamo_generation.py#L220) |

## Why it is worth changing

The placeholder does not stand for anything. In each of those branches there is no server at all, so there is no per-rank slot to hold open — the list length carries no information. What it does do is make the value lie to a plain truth test: `[None]` is truthy, so "we have URLs" reads as true when there are none.

Callers have to know about that and work around it:

- [`_shard_base_urls`](https://github.com/NVIDIA-NeMo/RL/blob/e9a855ff4bc0a41967e9491c41de093527e123b6/nemo_rl/algorithms/single_controller_utils/setup.py#L768-L773) cannot write `if not urls`; it needs `if not any(urls)`.
- [grpo.py:1543](https://github.com/NVIDIA-NeMo/RL/blob/e9a855ff4bc0a41967e9491c41de093527e123b6/nemo_rl/algorithms/grpo.py#L1543) prints the count, so a sync run reports `Reserved 1 vLLM server URLs: [None]`.

It also forces the type to be wider than the values ever are. Every one of these lists is either all real URLs or all `None`s — never mixed. If the "none available" case returned `[]`, the type could be `list[str]` instead of `list[Optional[str]]`, and callers downstream would not have to carry the `Optional` around. This came up while reviewing #3367, where a function signature was widened to `list[Optional[str]]` to match the producer's declared type even though a `None` cannot reach that code path.

## Suggested change

Make the three backends match Megatron: return `[]` when there is no HTTP server to report, and narrow `dp_openai_server_base_urls` to `list[str]`.

Worth checking while doing it:

- [`tests/unit/models/generation/test_dynamo_generation.py:167`](https://github.com/NVIDIA-NeMo/RL/blob/e9a855ff4bc0a41967e9491c41de093527e123b6/tests/unit/models/generation/test_dynamo_generation.py#L167) asserts `== [None]` and would need updating.
- TRT-LLM's version is length `dp_size` rather than 1, so anything that reads the length in the no-server case should be checked before changing it.
- `_shard_base_urls` and the `any(...)` guards can be simplified once the sentinel is gone, but they stay correct either way, so they can be cleaned up separately.

Contributor guide

Open the contributing guide

Research direction

Start with the cited no-server branches in vllm_generation.py, trtllm_generation.py, and dynamo_generation.py, then inspect dp_openai_server_base_urls callers such as _shard_base_urls and grpo.py. Update the affected tests, including tests/unit/models/generation/test_dynamo_generation.py:167, and check TRT-LLM length handling. Done means no-server cases return [] consistently and the type and callers no longer require Optional URLs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.