microsoft / microsoft/amplifier

OpenAI root sessions spin: three "silent configuration" defects (effort shadowing, skills-visibility ignores enabled:false, delegate O(agents))

Open
#393 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.1k
Forks
261
Avg merge
3h 28m
Merged PRs (30d)
13

Description

OpenAI root sessions spin: three "silent configuration" defects

Summary

On a dense personal bundle, an OpenAI root session pinned to the flagship tier at reasoning_effort: xhigh reliably spins — it generates its own review loops, inflates verification far past the ask, and runs one user turn for hundreds of internal tool calls before checking in. Dropping effort xhigh → high fixes the immediate behavior.

But the effort value was the symptom. The deeper, reusable finding is that three independent places in Amplifier apply or inject configuration the user cannot see or turn off, and together they make an OpenAI root session hard to tame. Each is small; each is a "make the invisible visible / honor the off switch" fix. Filing them together because they share that theme and the same root story.

All line references are against the currently-installed builds (amplifier_foundation 1.0.0, amplifier_app_cli, amplifier_module_provider_openai 1.0.0, amplifier_module_tool_delegate, amplifier_module_tool_skills). Happy to split into per-repo issues if preferred.


Defect 1 — provider-config effort is silently shadowed, with no provenance

What happens. For a ROOT session, reasoning_effort resolution precedence is:

  1. settings.yaml / project .amplifier/settings.local.yaml config.providers[].config, merged over the bundle's providers[].config by matching idsettings wins (runtime/config.py _apply_provider_overridesmerge_module_itemsdeep_merge, child overlay wins; lib/merge_utils.py).
  2. Bundle providers[].config — loses to any same-id settings entry.
  3. Routing matrix candidate config — does not govern the root's pinned model (matrix only routes delegated sub-agents; commands/run.py --model/--provider handling never consults the matrix).
  4. provider-openai module default — ships None for the current family (_capabilities.py).

Why it bites. A user who sets reasoning_effort: medium in a bundle overlay, then pins a model, gets whatever settings.yaml says (often xhigh written by the provisioning wizard) — with no indication the bundle value was overridden. The bundle knob is a dead key. Users reasonably believe they've turned effort down when they haven't.

Proposed fix (small, high value). This is a visibility defect, not a precedence bug — settings-wins is correct (it carries secrets/identity). amplifier_app_cli already walks config["providers"] in effective_config.py; add per-key provenance (bundle vs overrides.<id> vs config.providers[]) and surface it in amplifier provider show. Silent shadowing is the actual defect.


Defect 2 — the skills-visibility block ignores tool-skills.config.visibility.enabled: false

What happens. The anchors bundle mounts tool-skills with visibility.enabled: false (bundles/anchors/bundle.md). On a session composed from that base, the <system-reminder source="hooks-skills-visibility"> block — a full listing of ~46 skills with descriptions — still appears in the system context every request (~9–10 KB). The module default is True (tool_skills hook), and the false from the mounted config does not appear to suppress the injection.

Evidence. Reproduced on a live session whose composed stack sets visibility.enabled: false at the anchors layer with nothing re-enabling it downstream (verified no overlay in the stack sets visibility). The block is nonetheless present.

Why it matters. On OpenAI at high effort, instruction density is a multiplier — the model spends reasoning budget satisfying an injected catalog it was configured not to see. ~10 KB of every request, unkillable by the documented switch.

Ask. Confirm whether hooks-skills-visibility honors visibility.enabled: false from the composed mount config; if not, that's the fix. (Repro: compose any bundle on anchors, start a session, inspect the system prompt for the skills-visibility reminder.)


Defect 3 — the delegate tool schema inlines every agent's full description, O(agents) per request

What happens. tool-delegate emits " - {name}: {description}" for every registered agent, full text, unconditionally, into the delegate tool schema — which is re-sent on every request. Cost scales linearly with the number of registered agents, whether or not they're ever called. On rosters that have grown, this is a large, permanent per-request tax.

Proposed fix. Add a listing mode/threshold to tool-delegate (e.g. features.agent_listing: {mode: full|summary|lazy, max_inline: N}): above the threshold, emit name: <first sentence> and expose the full roster via an explicit delegate(list_agents=true) call. This is a genuine scaling property, not a per-bundle config choice. (Owner: @sadlilas.)

Related enabler. To let a general guard warn when a session is pinned to a spin-inducing effort, expose the resolved reasoning_effort on ProviderInfo / provider get_info() (provider-openai resolves it internally at mount but does not surface it). Small addition; unblocks a portable "effort guard" hook.


Suggested ownership

  • amplifier-foundation — config-resolution provenance surface (Defect 1 backend); ProviderInfo.reasoning_effort (Defect 3 enabler).
  • amplifier-app-cliamplifier provider show provenance display (Defect 1).
  • amplifier-module-tool-skills — honor visibility.enabled: false (Defect 2).
  • amplifier-module-tool-delegate — agent-listing threshold (Defect 3). @sadlilas

Why one issue

All three are the same failure class: configuration or context that the system applies/injects without the user being able to see it or turn it off. Fixing them is what turns "OpenAI sessions spin unpredictably" into "OpenAI sessions do what the config says." Each fix is independently small and independently shippable.

cc @bkrabach @sadlilas

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Split the report by ownership, then read runtime/config.py, lib/merge_utils.py, effective_config.py, bundles/anchors/bundle.md, and the tool-skills and tool-delegate entry points named in the issue. Reproduce the composed configuration and inspect the provider display, skills-visibility reminder, and delegate schema. Done means each reported setting is visible or honored, and delegate context growth is addressed with tests or a documented result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, developer-experience, performance, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.