galaxyproject / galaxyproject/loom

Context-fill indicator: use Pi's getContextUsage() for an authoritative window % (Option B)

Open
#165 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
14
Forks
12
Avg merge
6d 5h
Merged PRs (30d)
17

Description

## Background

PR #164 added a context-window fill indicator to the Orbit footer ("Option A" — renderer-only). It works by approximating the current request size client-side from the latest turn's streamed `usage` (`input + cacheRead + cacheWrite`) and dividing by the model's `contextWindow` as reported by pi-ai's registry (forwarded through the `models:list-all` IPC).

That's a good, low-cost first cut, but it has two accuracy gaps:

1. **The denominator can be wrong.** The registry `contextWindow` is the *API model's* maximum. For `openai-codex` models (e.g. `gpt-5.5` = 272000) the ChatGPT-subscription endpoint may enforce a smaller real limit, so the bar can under-report — showing comfortable headroom right before a `context_length_exceeded`. (This is the exact failure that motivated the indicator.)
2. **The numerator is a client-side reconstruction.** It sums streamed usage fields and does not account for the parts of the request the renderer never sees as "usage" — e.g. the full system prompt, tool definitions, and pi's compaction `reserveTokens` headroom. It tracks the *last* request, not what the *next* one will cost.

## Proposal — Option B

Drive the indicator from Pi's own context accounting instead of the renderer's approximation.

Pi exposes, on the extension context:

```ts
getContextUsage(): ContextUsage | undefined;
interface ContextUsage {
tokens: number | null; // estimated context tokens (null right after compaction)
contextWindow: number;
percent: number | null; // usage as % of the window
}
```

This is purpose-built for exactly this indicator and is the authoritative number — it's the same accounting Pi uses to decide when to auto-compact (`shouldCompact(contextTokens, contextWindow, settings)`), so the bar would agree with when compaction actually fires.

### Sketch

- **Loom (extension):** on `turn_end` (and after `session_compact`), call `ctx.getContextUsage()` and push `{ tokens, contextWindow, percent }` to the shell over the existing UI bridge (a small status/widget message). Handle the `null` cases (right after compaction) gracefully.
- **Orbit (renderer):** render the pushed `percent`/`tokens`/`contextWindow` directly, replacing the client-side `contextTokens / CONTEXT_WINDOWS[...]` computation from #164. Keep the same bar + color-threshold UI.
- Once Option B is the source of truth, the `CONTEXT_WINDOWS` map and the `contextWindow` IPC plumbing added in #164 can be retired (or kept only as a pre-first-turn fallback).

### Benefits

- Correct denominator per provider/endpoint (no registry-vs-reality drift).
- Numerator includes everything Pi counts (system prompt, tools, reserve), so the bar matches the real overflow boundary and the auto-compaction trigger point.
- Single source of truth shared with compaction.

## Acceptance criteria

- [ ] Indicator value comes from `ctx.getContextUsage()`, pushed from the extension, not reconstructed in the renderer.
- [ ] Handles `tokens === null` / `percent === null` (e.g. immediately post-compaction) without flicker or NaN.
- [ ] Updates at least once per turn and after compaction.
- [ ] On the `openai-codex` / subscription path, the reported % reflects the real limit Pi is accounting against (no silent under-report).
- [ ] #164's renderer-side `contextTokens` math and (optionally) the `contextWindow` IPC/`CONTEXT_WINDOWS` fallback are removed or demoted to a pre-first-turn placeholder.

## References

- PR #164 — context-fill indicator (Option A), where this was flagged as the follow-up.
- PR #154 — `/compact` + cache-stable system prompt (related context-management work).
- Pi API: `ExtensionContext.getContextUsage()` / `ContextUsage` in `@earendil-works/pi-coding-agent`.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.