NVIDIA-NeMo / NVIDIA-NeMo/Gym

fix(responses): downconverter rejects include on presence, so Codex CLI cannot call any chat-backed model server

Open Beginner friendly
#3,312 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.2k
Forks
349
Avg merge
1d 23h
Merged PRs (30d)
324

Description

Describe the bug

ResponsesConverter.responses_to_chat_completion_create_params (nemo_gym/responses_converter.py:173-191 at 86e2252f5) rejects a Responses request whenever the include key is present, whatever its value:

unsupported_fields = sorted(
    {"background", "context_management", "conversation", "include", "max_tool_calls",
     "previous_response_id", "prompt", "truncation"}
    & responses_create_params.keys()
)
if unsupported_fields:
    raise NotImplementedError(...)

The Codex CLI sends include on every request: at Gym's pinned codex_version: 0.144.4, ResponsesApiRequest (codex-rs/codex-api/src/common.rs) declares pub include: Vec<String> without skip_serializing_if, and codex-rs/core/src/client.rs:874-878 fills it with ["reasoning.encrypted_content"] or [].
For a model served through Gym it is [].

So every Codex call to a chat-backed Gym model server (vllm_model, azure_openai_model, inference_provider) ends in a terminal response.failed; Codex retries five times, then fails the turn.
The showcase config resources_servers/reasoning_gym/configs/reasoning_gym_codex_agent_model_server.yaml and the codex_agent README section "Against a Gym model server" cannot complete one model call.

Steps/Code to reproduce bug

  1. No server needed:
from nemo_gym.responses_converter import ResponsesConverter
from nemo_gym.responses_streaming import sanitize_streaming_responses_body, validate_streaming_responses_params

body = {"model": "x", "stream": True, "store": False, "include": [],
        "input": [{"type": "message", "role": "user", "content": [{"type": "input_text", "text": "2+2?"}]}]}
cleaned, _ = sanitize_streaming_responses_body(body)
params = validate_streaming_responses_params(cleaned)
ResponsesConverter(return_token_id_information=False).responses_to_chat_completion_create_params(params)
# NotImplementedError: Responses request field(s) ['include'] have no Chat Completions representation, ...

The same happens with "include": ["reasoning.encrypted_content"].

  1. Codex CLI 0.144.4 with the config.toml that responses_api_agents/codex_agent/app.py generates, against any chat-backed model server:
{"type":"error","message":"Reconnecting... 5/5 (stream disconnected before completion: Responses request field(s) ['include'] have no Chat Completions representation, so this request cannot be downconverted. Route it to a model server that passes Responses through.)"}
{"type":"turn.failed","error":{"message":"stream disconnected before completion: Responses request field(s) ['include'] ..."}}

The server log shows six rejections per turn (one call plus five retries), each with include: [].
Reproduced on vLLM-served Qwen3-8B and, without a GPU, on the canned SimpleResponsesAPIModel below, which runs the real ResponsesConverter.

GPU-free chat-backed model server used for step 2
# fake_gym_model_server.py: python fake_gym_model_server.py 8099
import sys
from unittest.mock import MagicMock

import uvicorn
from fastapi import Body
from openai.types.completion_usage import CompletionUsage

from nemo_gym.base_responses_api_model import BaseResponsesAPIModelConfig, SimpleResponsesAPIModel
from nemo_gym.openai_utils import (
    NeMoGymChatCompletion, NeMoGymChatCompletionCreateParamsNonStreaming, NeMoGymChatCompletionMessage,
    NeMoGymChoice, NeMoGymResponse, NeMoGymResponseCreateParamsNonStreaming,
)
from nemo_gym.responses_converter import ResponsesConverter
from nemo_gym.server_utils import ServerClient


class CannedChatBackedModel(SimpleResponsesAPIModel):
    config: BaseResponsesAPIModelConfig
    model_config = {"arbitrary_types_allowed": True}

    async def chat_completions(self, body: NeMoGymChatCompletionCreateParamsNonStreaming = Body()) -> NeMoGymChatCompletion:
        return NeMoGymChatCompletion(
            id="chatcmpl-canned", created=0, model=body.model or "canned", object="chat.completion",
            choices=[NeMoGymChoice(index=0, finish_reason="stop",
                                   message=NeMoGymChatCompletionMessage(role="assistant", content="4"))],
            usage=CompletionUsage(prompt_tokens=11, completion_tokens=1, total_tokens=12),  # totals only, like vLLM
        )

    async def responses(self, body: NeMoGymResponseCreateParamsNonStreaming = Body()) -> NeMoGymResponse:
        print("REQUEST include:", body.model_dump(exclude_unset=True, exclude_none=True).get("include"), file=sys.stderr, flush=True)
        converter = ResponsesConverter(return_token_id_information=False)
        chat_params = converter.responses_to_chat_completion_create_params(body)  # the vllm_model path
        return converter.chat_completion_to_response(responses_create_params=body,
                                                     chat_completion=await self.chat_completions(chat_params))


if __name__ == "__main__":
    port = int(sys.argv[1]) if len(sys.argv) > 1 else 8099
    server = CannedChatBackedModel(
        config=BaseResponsesAPIModelConfig(host="127.0.0.1", port=port, entrypoint="", name="canned"),
        server_client=MagicMock(spec=ServerClient, global_config_dict={}),
    )
    uvicorn.run(server.setup_webserver(), host="127.0.0.1", port=port, log_level="warning")

Expected behavior

include only adds optional fields to the response; it does not change what the model is asked.
An empty list, or an entry a Chat Completions backend cannot produce anyway (reasoning.encrypted_content), should be ignored so the request downconverts; entries that would need backend support (message.output_text.logprobs) can keep failing loudly.
The fix belongs in the converter, not in sanitize_streaming_responses_body: the passthrough servers (openai_model, litellm_model) forward include to providers that honour it.

Configs
gym env start --resources-server reasoning_gym/reasoning_gym_codex_agent_model_server --model-type vllm_model, unmodified.

Environment details

  • NeMo Gym main @ 86e2252f5
  • Linux x86_64, Python 3.13.14, openai==2.44.0, Codex CLI 0.144.4 (Gym's pin)
  • vLLM-served Qwen3-8B on A100

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in nemo_gym/responses_converter.py at ResponsesConverter.responses_to_chat_completion_create_params, especially lines 173-191, and run the provided include=[] reproduction. Verify that empty or unsupported-by-Chat-Completions include entries no longer block downconversion, while entries requiring backend support such as message.output_text.logprobs still fail; confirm the Codex model-server path can complete a request.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.