Partial multimodal overrides erase catalog image defaults

Partial multimodal overrides erase catalog image defaults

Summary

When a model has catalog multimodal defaults, an input-only override such as multimodal: { input: ["text", "image"] } keeps the input modalities but clears other default image constraints such as maxImagesPerRequest, supportedImageMimeTypes, and imageDetail.

This happens because parseMultimodal() spreads the defaults and then assigns optional fields even when the override did not provide those fields.

Affected revision

Observed on main at 9ad10eb0e1ed75f864ca8cbda7f659f7c3b163e9.

Affected code

• src/model/config/parseModelConfig.ts:267 computes the default multimodal constraints.
• src/model/config/parseModelConfig.ts:291-307 spreads the defaults but overwrites optional fields with values read from the partial override.
• src/model/request/validateModelRequest.ts consumes model.multimodal during request validation.

Reproduction

From the repository root:

cat > repro-multimodal-defaults.mts <<'EOF'
import { parseModelConfig } from "./src/model/config/parseModelConfig.ts";

const env = { OPENAI_API_KEY: "test-key" };

const baseline = parseModelConfig({
  providers: {
    openai: {
      models: {
        "gpt-4o-mini": {},
      },
    },
  },
}, { env });

const withInputOnly = parseModelConfig({
  providers: {
    openai: {
      models: {
        "gpt-4o-mini": {
          multimodal: { input: ["text", "image"] },
        },
      },
    },
  },
}, { env });

console.log(JSON.stringify({
  baseline: baseline.providers.openai.models["gpt-4o-mini"].multimodal,
  withInputOnly: withInputOnly.providers.openai.models["gpt-4o-mini"].multimodal,
}, null, 2));
EOF

pnpm exec tsx repro-multimodal-defaults.mts
rm repro-multimodal-defaults.mts

Observed output:

{
  "baseline": {
    "input": ["text", "image"],
    "maxImagesPerRequest": 20,
    "supportedImageMimeTypes": ["image/jpeg", "image/png", "image/gif", "image/webp"],
    "imageDetail": "auto"
  },
  "withInputOnly": {
    "input": ["text", "image"]
  }
}

Expected behavior

An input-only override should not clear unrelated catalog defaults. If input is the only overridden field, the remaining constraints should continue to come from the catalog/protocol defaults.

Actual behavior

The partial override removes the default image limit, MIME allowlist, and image detail setting.

Impact

Users can unintentionally weaken or change multimodal validation by setting only the supported input modalities. Since request validation consumes model.multimodal, clearing these defaults can make routing/validation less constrained than the catalog model definition.

Existing coverage

I could not find an existing issue or pull request covering partial multimodal overrides erasing catalog image defaults.

Suggested fix

Only assign optional multimodal fields when the corresponding key is present in the raw override, or merge a filtered object that omits undefined values. Add a regression test where an input-only override for openai/gpt-4o-mini preserves the same image constraints as the catalog default.

Suggested tests

• An input-only override for openai/gpt-4o-mini preserves maxImagesPerRequest, supportedImageMimeTypes, and imageDetail.
• Explicitly provided optional multimodal fields still override defaults.
• Invalid optional multimodal fields still raise the existing config errors.

---

Submitted with Codex.

Start in src/model/config/parseModelConfig.ts at the cited default computation and parseMultimodal() range, then run the provided pnpm exec tsx reproduction from the repository root. Check how src/model/request/validateModelRequest.ts consumes model.multimodal, and add regression coverage showing input-only overrides preserve catalog image constraints while explicit and invalid optional fields retain their existing behavior.