ng_prepare_data fails for configs with multiple model servers (e.g. user_model)

Describe the bug

ng_prepare_data fails when the agent config references additional model servers beyond policy_model (e.g. user_model for the multi-turn agent). The root cause is that NO_MODEL_GLOBAL_CONFIG_DICT injects a dummy policy_model with responses_api_models: {dummy_model: {entrypoint: app.py}}. When the user also passes a model config that defines policy_model (e.g. openai_model_with_user.yaml), Hydra deep-merges the two, resulting in 2 entries under policy_model.responses_api_models — which violates the max_length=1 constraint on ResponsesAPIModelServerTypeConfig.

Even without passing a model config, ng_prepare_data fails because the dummy only provides policy_model, so agent configs that reference a user_model server hit an assertion error.

Steps/Code to reproduce bug

# Fails: deep-merge creates 2 entries under policy_model.responses_api_models
ng_prepare_data "+config_paths=[resources_servers/example_multi_turn/configs/example_multi_turn.yaml,responses_api_models/openai_model/configs/openai_model_with_user.yaml]" \
    +output_dirpath=resources_servers/example_multi_turn/data \
    +mode=example_validation

# Also fails: user_model server ref not found in dummy config
ng_prepare_data "+config_paths=[resources_servers/example_multi_turn/configs/example_multi_turn.yaml]" \
    +output_dirpath=resources_servers/example_multi_turn/data \
    +mode=example_validation

Workaround — manually inject a dummy user_model on the CLI:

ng_prepare_data "+config_paths=[resources_servers/example_multi_turn/configs/example_multi_turn.yaml]" \
    +output_dirpath=resources_servers/example_multi_turn/data \
    +mode=example_validation \
    "+user_model={responses_api_models: {dummy_model: {entrypoint: app.py}}}" \
    +user_base_url="" +user_api_key="" +user_model_name=""

Expected behavior

ng_prepare_data should work with agent configs that reference additional model servers (e.g. user_model, judge_model) without requiring manual CLI overrides. The dummy config injection should either handle arbitrary model server refs or use a replace-merge strategy instead of deep-merge.

Configs

NO_MODEL_GLOBAL_CONFIG_DICT in nemo_gym/global_config.py:

NO_MODEL_GLOBAL_CONFIG_DICT: ClassVar[DictConfig] = DictConfig(
    {
        POLICY_BASE_URL_KEY_NAME: "",
        POLICY_API_KEY_KEY_NAME: "",
        POLICY_MODEL_NAME_KEY_NAME: "",
        POLICY_MODEL_KEY_NAME: {"responses_api_models": {"dummy_model": {"entrypoint": "app.py"}}},
    }
)

The max_length=1 constraint is in nemo_gym/config_types.py:

class ResponsesAPIModelServerTypeConfig(BaseServerTypeConfig):
    responses_api_models: Dict[str, BaseRunServerTypeConfig] = Field(min_length=1, max_length=1)

Environment details

• Any OS / Python version — issue is in config parsing logic, not platform-specific

Additional context

This affects any resources server paired with a multi-model agent (e.g. multi_turn_agent with user_model_server, or servers using a judge_model). The ng_test_all CI step runs data validation, so these servers will fail CI until the workaround is applied or the framework is fixed.

Start in nemo_gym/global_config.py and nemo_gym/config_types.py, then reproduce both ng_prepare_data commands from the issue. Verify that configs referencing user_model or other model servers load without manual CLI overrides and that the resulting validation passes without violating the single-entry constraint.