mpfaffenberger / mpfaffenberger/code_puppy

ModelFactory.load_config() re-reads/parses up to 6 JSON files + fires plugin hooks on every call (no caching) — invoked 5+ times per settings resolution; plugin providers loaded as import-time side effect

Open
#426 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
814
Forks
278
Avg merge
2d 5h
Merged PRs (30d)
76

Description

Problem

ModelFactory.load_config() (code_puppy/model_factory.py lines ~390-525) performs, on every call:

  • read + parse bundled models.json
  • stat + read + parse up to 5 overlay files (extra_models.json, ChatGPT/Claude/Gemini/Copilot OAuth model files)
  • fire the load_models_config plugin hook
  • re-read + re-parse bundled models.json a second time for the description overlay pass
  • fire the load_model_descriptions plugin hook

And it has many high-frequency callers:

  • config.get_model_context_length() → called by get_protected_token_count() (compaction checks every turn)
  • config.model_supports_setting() → called once per setting inside get_effective_model_settings() (so N config-loads per settings resolution)
  • config._validate_model_exists(), _default_model_from_models_json()
  • provider_credentials._load_merged_model_config()all_required_env_vars()
  • model_factory.make_model_settings() itself

A single make_model_settings() call can trigger load_config() 5+ times; a single agent turn easily reaches dozens of full multi-file JSON parses. Combined with get_value()'s per-call cfg re-parse (#399) this is the bulk of avoidable per-turn I/O.

Also: _load_plugin_model_providers() runs at module import time (line ~57) — a module-level side effect that fires plugin hooks whenever anything imports model_factory, making import order matter and complicating tests.

Suggested fix

Cache the merged config with explicit invalidation, mirroring the existing clear_model_cache() pattern:

_config_cache: Dict[str, Any] | None = None

@staticmethod
def load_config(*, refresh: bool = False) -> Dict[str, Any]:
    global _config_cache
    if _config_cache is not None and not refresh:
        return _config_cache
    config = ModelFactory._load_config_uncached()
    _config_cache = config
    return config

Invalidate from config.clear_model_cache() (already the designated "models.json changed" hook) and from /add_model & OAuth flows that write the overlay files. Reuse the first bundled-models.json parse for the description overlay instead of re-reading the file. Move _load_plugin_model_providers() to first use (lazy) instead of import time.

Filed by Zen Reviewer A (code-puppy-60635a)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in code_puppy/model_factory.py around ModelFactory.load_config() and the import-time _load_plugin_model_providers() call. Trace config.clear_model_cache(), /add_model, OAuth overlay writers, and the listed high-frequency callers before changing cache ownership and invalidation. Done means repeated loads avoid redundant parsing and hooks, description loading reuses the bundled parse, plugin providers load lazily, and invalidation still reflects file changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.