microsoft / microsoft/agent-framework
Python: perf(foundry-hosting): FoundryAgentSessionStore rebuilds FoundryStateStore (credential + metadata round-trip) on every request
@eavanvalkenburg is already working on this.
Since Sep 14, 2026.
- Dominant language
- Python
- Stars
- 13.6k
- Forks
- 2.3k
- Avg merge
- 2d 45m
- Merged PRs (30d)
- 358
Description
### Describe the bug / perf issue
`FoundryAgentSessionStore` (the default agent-session `SessionStore` for hosted MAF agents) rebuilds its backing `FoundryStateStore` on **every** `get`/`set`/`delete`. Each `_get_store()` call goes through `FoundryStateStore.get_or_create("agent_sessions", user_isolation=True)`, which:
1. constructs a fresh credential (empty token cache → a new managed-identity token fetch), and
2. issues an `agent_sessions` **metadata round-trip** (`GET`/`POST state_stores`) *before* the actual item operation.
Because the hosting infra calls `set()` in the `finally` of every Responses request, this redundant credential + metadata work lands on the **critical path of every request**.
### Component
`python` / `foundry_hosting` — `FoundryAgentSessionStore` (`_state_store.py`).
### Expected behavior
The agent-session scope (`"agent_sessions"`, `user_isolation=True`) is identical for every request, so the backing store should be resolved once and reused for the process lifetime. Only the real item `GET`/`PUT`/`DELETE` should remain on the hot path — not a per-request credential build + metadata round-trip.
### Measured impact
500 cold + 500 warm streaming requests per agent, concurrency 50, private/VNet Foundry project, `gpt-4o-mini`, 0 errors / 0 throttled. Same-conditions A/B, baseline vs cached Responses agent (ms, p50):
| Metric | Baseline | Cached | Δ |
|---|---|---|---|
| WARM TTLB p50 | 7,736 | 7,464 | **−272 ms (−3.5%)** |
| WARM TTFB p50 | 5,680 | 5,683 | ~flat |
The win is the elimination of the per-request credential + metadata round-trip; the remaining warm latency is the model first-token floor (~5.7 s TTFB), unaffected by this change.
### Proposed fix
Cache one **per-event-loop, per-scope** `FoundryStateStore` for the agent-session scope (guarded by a per-loop lock with double-checked init; keyed by the running loop via a `WeakKeyDictionary` so a closed loop's store is GC'd, and keyed by scope so subclasses overriding `DEFAULT_ROOT_SCOPE` are isolated). Item `get`/`set`/`delete` semantics stay byte-for-byte identical; checkpoint / function-approval stores are left untouched (not on the per-request hot path).
Fix implemented in PR #8178 (closed pending this tracking issue).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.