zai-org / zai-org/feedback

[Bug] Custom-provider models (deepseek/qwen) listed but not selectable — reasoning-effort validated against stale bundled catalog

Open
#623 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

priority: P2
Dominant language
No language data
Stars
22
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Pre-submission checklist · Pre-submission checklist
  • I have searched existing issues and confirmed this isn't a duplicate / I have searched existing issues and confirmed this isn't a duplicate.
  • I have read CONTRIBUTING.md / I have read CONTRIBUTING.md.
Problem category · Category

Model settings / switching · Model config

Affected agent framework · Agent framework

Not framework-specific · Not framework-specific (affects both ZCode Agent model picker and subagent model overrides)

Severity · Severity

Major (feature unusable / major: custom-provider models are listed but cannot be selected or used at all)

Reproducibility · Reproducibility

Always · Always

Description · Description

Models added under a custom provider (tested: DeepSeek via builtin:deepseek with baseURL https://api.deepseek.com/anthropic, API key configured; same with an Alibaba token-plan anthropic endpoint) appear in the model picker but cannot be selected, and cannot be used as subagent models via ~/.zcode/v2/agents-state.json (builtInModelOverrides). Every attempt fails with:

Reasoning effort "X" is not supported by builtin:deepseek/deepseek-flash   # X in {high, low, off} — all rejected

Root cause (verified): the reasoning-effort validation is performed against the bundled model catalog (/opt/ZCode/resources/model-providers/models_catalog_china_llm_zcode_2026-06-03.json), which is older than the live API. The live API and the bundled catalog disagree on model names, so custom-provider models can never pass validation:

  • live API GET https://api.deepseek.com/models (with the configured key) → ["deepseek-flash", "deepseek-v4-pro"]deepseek-flash exists in the API
  • bundled catalog knows only deepseek-v4-flash / deepseek-v4-prono deepseek-flash
  • additionally, manually adding a reasoning.levels block to the custom model entry in ~/.zcode/v2/config.json is silently stripped on the next app start (config normalization), so it cannot be used as a workaround

The same models work perfectly when called directly (verified outside ZCode with the same key):

  • GET https://api.deepseek.com/models → HTTP 200
  • POST https://api.deepseek.com/anthropic/v1/messages (x-api-key, anthropic-version 2023-06-01) → HTTP 200, model=deepseek-flash
  • Alibaba token-plan anthropic endpoint → HTTP 200, model=qwen3.8-max
Steps to reproduce · Steps to reproduce
  1. Settings → Model Settings → add a custom provider (e.g. DeepSeek, base URL https://api.deepseek.com/anthropic, valid API key) and a custom model named exactly like an API model, e.g. deepseek-flash.
  2. Open the model picker (Ctrl+M): the model is listed, but selecting it is rejected (nothing happens / cannot be chosen).
  3. Alternatively set a subagent override: ~/.zcode/v2/agents-state.json"builtInModelOverrides": {"Explore": "builtin:deepseek/deepseek-flash"} (+ restart), then trigger any subagent spawn.
  4. Observe: Reasoning effort "high" is not supported by builtin:deepseek/deepseek-flash — and the same failure for "low", and even with no builtInThoughtLevelOverrides entry (session-inherited effort is rejected as well).
  5. Note: using the bundled-catalog name builtin:deepseek/deepseek-v4-flash (after app restart) also fails — either effort rejection, or (in a later attempt with another model name) Model provider is not configured: builtin:deepseek even though the provider is configured in ~/.zcode/v2/config.json.
Expected vs actual · Expected vs actual behavior

Expected: models added via custom providers are selectable in the picker and usable as subagent models. If a provider/model does not declare reasoning levels, the effort parameter should be omitted in the upstream request (the DeepSeek API accepts requests without reasoning parameters) instead of failing local validation.

Actual: any reasoning effort value is rejected against the stale bundled catalog before a request is made; custom-provider models are effectively unusable, while the exact same models work when called directly with the same credentials.

Notes
  • Requested effort levels for deepseek-v4-flash per the bundled catalog itself: off, high, max (default max) — yet off was also rejected for a subagent, so subagent-level validation seems to use a different (narrower) set than the catalog.
  • After app restart the app normalizes ~/.zcode/v2/config.json and strips unknown fields from custom model entries — please treat this file as app-managed when reproducing.
  • Version tested: 3.11.2 (latest available on the download page at reporting time). Catalog file dated 2026-06-03.
Logs & screenshots · Logs & screenshots

Available on request (local validation failures do not appear in the network logs; the validation happens before any HTTP request).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure using ~/.zcode/v2/config.json, ~/.zcode/v2/agents-state.json, and the bundled model catalog at /opt/ZCode/resources/model-providers/models_catalog_china_llm_zcode_2026-06-03.json. Trace reasoning-effort validation and custom-model config normalization, then verify that valid custom-provider models can be selected and that requests without declared reasoning levels omit the effort parameter. Add regression coverage for picker and subagent model selection if the repository provides corresponding tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
json
Domain
ai, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.