1jehuang / 1jehuang/jcode

[Bug] Model ID case normalization causes exact-match failure and silent fallback to wrong model

Open
#705 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

autonomous: likely bug priority: medium triage: reproducible
Dominant language
Rust
Stars
19.9k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
30

Description

Summary

When jcode populates the account model catalog from a custom OpenAI-compatible endpoint, it forcefully lowercases all model IDs. However, model validation uses a case-sensitive exact match against that lowercased set, causing any mixed-case model ID (e.g., GPT5-Mitarbeitende) to fail validation. provider_init logs this failure as a warning (invisible under --quiet), and model resolution silently falls back to the alphabetically first catalog entry.

Step-by-Step Failure Chain
  1. Catalog Lowercasing: In crates/jcode-base/src/provider/models.rs:569-577, populate_account_models_for_scope normalizes every catalog model ID through normalize_model_id -> to_ascii_lowercase() before inserting it into the catalog set.
  2. Case-Sensitive Validation Failure: In crates/jcode-provider-openai-runtime/src/openai_provider_impl.rs:693-697, set_model validates requested model IDs with an exact, case-sensitive match (known == model). A requested ID like GPT5-Mitarbeitende fails comparison against the lowercased catalog entry gpt5-mitarbeitende.
  3. Silent Warning: In src/cli/provider_init.rs:1798, provider.set_model failures are treated as non-fatal warnings (warn!("Failed to set initial model: {}", e)). This warning is entirely omitted if the user is running under --quiet.
  4. Phantom Model Substitution: In crates/jcode-provider-openai-runtime/src/lib.rs:1263, when set_model fails, model_id() falls back to get_best_available_openai_model(), which selects the alphabetically first catalog entry (e.g., gpt5-mini-mitarbeitende).
Net Impact

Requesting a valid mixed-case model like GPT5-Mitarbeitende silently sends gpt5-mini-mitarbeitende (in lowercase) on the wire. This causes silent model substitution and unexpected HTTP 404 errors from custom gateways, with no indication to the user that their requested model was rejected. There is no standard dictating that OpenAI-compatible endpoints must use lowercase models.

Proposed Fix
  1. Preserve Original Case: Stop forcefully lowercasing model IDs in populate_account_models_for_scope. The catalog should store the exact strings returned by the provider's /models endpoint.
  2. Flexible Validation & Transmission: In set_model, when validating the user's requested model against the catalog:
    • Attempt an exact case-sensitive match first.
    • If that fails, attempt a case-insensitive fallback (for user CLI convenience).
    • In either case, always adopt the exact casing from the catalog for internal state and wire transmission.
  3. Hard Error on Failure: Treat set_model failures during CLI initialization as fatal errors rather than silent warnings to prevent phantom model swaps.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with populate_account_models_for_scope in crates/jcode-base/src/provider/models.rs:569-577, then trace set_model in crates/jcode-provider-openai-runtime/src/openai_provider_impl.rs:693-697 and model_id in crates/jcode-provider-openai-runtime/src/lib.rs:1263. Review the provider initialization path at src/cli/provider_init.rs:1798. Done means mixed-case catalog IDs validate, preserve their catalog casing for transmission, and initialization does not silently substitute another model.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.