mpfaffenberger / mpfaffenberger/code_puppy
config.py model resolution guesses on failure: _validate_model_exists caches True on exceptions; vision default falls back to hardcoded 'gpt-4.1' that may not exist
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 814
- Forks
- 278
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 76
Description
Problem
Two related robustness issues in code_puppy/config.py's model resolution, both "in the face of ambiguity, guess" violations:
1. _validate_model_exists() caches and returns True on any failure (lines ~491-512)
except Exception:
# If we can't validate, assume it exists to avoid breaking things
_model_validation_cache[model_name] = True
return True
A transient failure in ModelFactory.load_config() (corrupted extra_models.json, plugin exception) makes every model name "valid" — and the wrong answer is then cached for the process lifetime, so even after the underlying problem resolves, get_global_model_name() keeps returning a stored model that may not exist, and the user gets a confusing downstream ValueError: Model 'X' not found in configuration from the factory instead of the intended graceful fallback. At minimum don't cache the failure result; better, log the exception.
2. _default_vision_model_from_models_json() hardcodes "gpt-4.1" fallback (lines ~450-489)
When no model config is loadable it returns the literal \"gpt-4.1\" — a model that is not in the user's catalog (the bundled models.json is 4 bytes / empty by design now). Callers will then fail looking up a phantom model. The non-vision sibling _default_model_from_models_json() was already fixed to return None and warn (per its docstring "Returning None lets callers detect the no-model state"); the vision variant should do the same instead of guessing a name. Also the preferred_candidates tuple (claude-4-0-sonnet, gemini-2.5-flash-preview-05-20) is a stale hardcoded heuristic that drifts from the actual catalog.
Suggested fix
except Exception:
logger.warning(\"Model validation failed for %s\", model_name, exc_info=True)
return True # permissive, but do NOT cache
and make _default_vision_model_from_models_json() return Optional[str], mirroring _default_model_from_models_json() (return None + one-shot warning), letting callers handle absence explicitly.
Filed by Zen Reviewer A (code-puppy-60635a)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in code_puppy/config.py with _validate_model_exists() and _default_vision_model_from_models_json(), then inspect their callers and the matching non-vision default helper. Verify that validation failures are not cached and that vision resolution can signal no available model without selecting a hardcoded name; done means the fallback behavior is explicit and avoids phantom catalog entries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100