WordPress / WordPress/php-ai-client

usingModelPreference() with an unrecognized model id silently falls back to the provider default (cost risk)

Open
#241 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
308
Forks
84
Avg merge
7d 21h
Merged PRs (30d)
2

Description

Summary

Requesting a model id that the resolved provider doesn't expose does not error — the SDK silently generates with the provider's default model. A typo, a stale id, or a non-dated alias silently routes traffic to a potentially far more expensive model (e.g. Haiku → Opus, ~15×) with no signal to the caller.

Repro

Anthropic provider registered (its catalog exposes claude-haiku-4-5-20251001, not a bare claude-haiku-4-5):

$result = wp_ai_client_prompt( 'Reply with one word: ok' )
    ->using_model_preference( 'claude-haiku-4-5' )   // not an id the provider exposes
    ->generate_text_result();

$result->get_model_metadata()->get_id(); // => "claude-opus-4-8"  (the provider default)

No exception, no WP_Error, no warning — just a silent upgrade to the default (most capable / priciest) model. The same happens for any unrecognized id.

Impact

Surprise billing. Demo/bundled agents and any config typo silently run on the priciest model, and the substitution is invisible unless the caller inspects the resolved model id on every response. (Downstream, openclaWP shipped agents configured for claude-haiku-4-5 that were silently served by Opus.)

Expected (any of)

  1. Throw / return WP_Error when a specific (non-auto) requested model id isn't available from the resolved provider.
  2. Resolve documented non-dated aliases (claude-haiku-4-5 → latest matching dated model), mirroring the providers' own alias behavior.
  3. At minimum, surface the substitution (warning / a flag on the result) so callers can detect "you asked for X, you got Y."

Notes

If the alias-resolution belongs in the individual provider plugins (e.g. ai-provider-for-anthropic) rather than the core client, happy to redirect — but the silent-fallback-to-default behavior on an unknown id looks like a core model-resolution concern. Reproduced on the php-ai-client bundled with WordPress 7.0.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at using_model_preference() and generate_text_result(), then trace how the requested model id is resolved against the provider catalog. Reproduce the unknown-id case and inspect the resolved model metadata. Done means the chosen behavior for an unavailable specific id is implemented with an observable outcome and no unintended silent fallback.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.