galaxyproject / galaxyproject/loom
Evaluate Agent SDK credit path for Loom LLM client
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
## Context
On June 15, 2026, Anthropic split programmatic usage out of Claude subscription quotas into a separate monthly **Agent SDK credit** ($100/mo on Max 5x, more on higher tiers). The credit covers the `claude-agent-sdk` libraries (Python + TS), the `claude -p` headless CLI, the Claude Code GitHub Action, and "third-party apps that authenticate with your Claude subscription" -- but **not** raw Messages API calls authed with `ANTHROPIC_API_KEY`, which continue billing at pay-as-you-go rates. This issue is research-only: figure out whether Loom should change anything to put personal/dev usage on the credit, and document the constraints around multi-tenant subscription auth so we can decide where to invest. No code changes here -- implementation tickets get split out after triage.
Sources: [Anthropic support article](https://support.claude.com/en/articles/15036540-use-the-claude-agent-sdk-with-your-claude-plan), [Agent SDK overview](https://code.claude.com/docs/en/agent-sdk/overview).
## Current state
Loom does not call the Anthropic Messages API or the Claude Agent SDK directly. It delegates the entire LLM call path -- agent loop, tool dispatch, MCP server lifecycle, provider negotiation -- to **Pi** (`@mariozechner/pi-coding-agent@^0.73.0`, `package.json:37`), which is its own agent framework with its own extension system. This is the load-bearing finding for everything below.
- **Client/loop.** Pi owns the `while stop_reason == "tool_use"` loop. Loom registers tools into Pi via `registerPlanTools(pi)`, `registerExecutionCommands(pi)`, `registerTeamTools(pi)`, `registerSessionIndexTools(pi)` (`extensions/loom/index.ts:40-50`). No `anthropic.messages.create` or `claude_agent_sdk.query` anywhere in the repo.
- **Auth.** Per-user, per-process. Either (a) `~/.loom/config.json` `llm.apiKey` (plaintext in CLI, Electron `safeStorage`-encrypted in Orbit) or (b) any of a long list of env vars (`bin/loom.js:204-223`). Anthropic-relevant entries: `ANTHROPIC_API_KEY`, `ANTHROPIC_OAUTH_TOKEN`. Orbit injects the resolved key into the brain subprocess via `buildSecretEnv()` (`app/src/main/agent.ts:22-31`); the Orbit main process itself never holds the key.
- **MCP wiring.** Config-driven, not hardcoded. `bin/loom.js:144-190` (CLI) and the Orbit equivalent write `~/.pi/agent/mcp.json` based on whether Galaxy creds are present -- Galaxy MCP registers as `uvx galaxy-mcp>=1.4.0` with `GALAXY_URL`/`GALAXY_API_KEY` in its env, BRC Analytics is always-on. `pi-mcp-adapter@^2.4.0` (`package.json:42`) is the Pi-side reader.
- **Model selection.** `~/.loom/config.json` `llm.provider` / `llm.model`, overridable via `/model` slash command or CLI `--provider`/`--model` flags. Anthropic is the default. No `cache_control` headers anywhere in Loom code -- if prompt caching happens, Pi does it internally.
- **Concurrency.** One user per process. Orbit spawns one brain subprocess per window (`app/src/main/agent.ts:289`); CLI is one analysis per invocation. Session state maps to `cwd`; changing cwd in Orbit tears down and respawns the brain (`agent.ts:210-226`).
- **Token telemetry exists, but only in team_dispatch.** `extensions/loom/teams/tool.ts:72-78` and `extensions/loom/teams/dispatcher.ts:59-201` already pull `input/output/cacheRead/cacheWrite` off Pi's `msg.usage` and total it across proposer/critic turns. The single-agent path doesn't surface per-session usage anywhere central. (TODO(dannon): capture a representative session trace for the $/session estimate; can't be done from code alone.)
- **Pi's own provider list (verified against `packages/ai/src/providers/`).** Built-in providers in `register-builtins.ts`: `anthropic-messages`, `openai-completions`, `mistral-conversations`, `openai-responses`, `azure-openai-responses`, `openai-codex-responses`, `google-generative-ai`, `google-vertex`, `bedrock-converse-stream`. **No `claude-code` or `claude-agent-sdk` provider is built in.** But Pi has an extension system (`pi install npm:`) that lets third parties register new providers, and that's where the interesting paths live.
## Constraints
### Credit coverage and rules ([support article](https://support.claude.com/en/articles/15036540-use-the-claude-agent-sdk-with-your-claude-plan))
> Claude Agent SDK usage in your own projects (Python or TypeScript) … `claude -p` command in Claude Code (non-interactive mode) … Claude Code GitHub Actions integration … Third-party apps that authenticate with your Claude subscription
> Credits belong to individual accounts. They can't be shared or pooled across teammates.
> When your monthly credit runs out, additional Agent SDK usage flows to extra usage at standard API rates -- but only if you've enabled extra usage. If extra usage isn't enabled, Agent SDK requests stop until your credit refreshes.
> Your credit resets at the start of each billing cycle. Unused credits don't roll over.
Credit amounts: Pro $20 · Max 5x $100 · Max 20x $200 · Team Standard $20 · Team Premium $100 · Enterprise (seat-based Premium) $200.
### Third-party policy ([Agent SDK overview](https://code.claude.com/docs/en/agent-sdk/overview))
> Unless previously approved, Anthropic does not allow third party developers to offer claude.ai login or rate limits for their products, including agents built on the Claude Agent SDK. Please use the API key authentication methods described in this document instead.
This sits in the SDK's "Set your API key" step, not a footnote. **It matters specifically for the bundle-this-into-Loom-and-ship-it-to-Galaxy-users path.** Personal install of a Pi extension on your own machine that uses your own Claude subscription is the user authenticating their own client to their own subscription -- Anthropic isn't policing that. But if Loom-as-a-service offered claude.ai login to end users, that's the gated path.
### What the docs don't say
- The `claude setup-token` / `CLAUDE_CODE_OAUTH_TOKEN` headless-auth flow is **not documented** in the support article, the SDK overview, or the Python SDK reference. The SDK's "Set your API key" step shows only `ANTHROPIC_API_KEY` plus Bedrock/Vertex/Azure variants. The OAuth-token flow exists in the Claude Code CLI but Anthropic doesn't currently surface it as a sanctioned "use your subscription headlessly" mechanism in SDK docs.
- No published RPM / concurrent-session caps beyond the $ cap.
- No documented behavior for "credit exhausted mid-session" beyond the binary "extra usage on → flows to API rates / off → requests stop until refresh."
### The buckets, post-June-15
| Path | Billing bucket | Loom-compatible today? |
|---|---|---|
| `ANTHROPIC_API_KEY` (Console) | Pay-as-you-go API rates | Yes -- default |
| Pi's own `/login` → `ANTHROPIC_OAUTH_TOKEN` | Interactive-subscription quota (claude.ai), **not** Agent SDK credit | Yes -- env var already passes through at `bin/loom.js:206`; gray area for multi-user, fine for personal |
| `claude-agent-sdk` library (Anthropic-sanctioned) | Agent SDK credit | Only via a Pi extension that uses the SDK as backend |
| `claude -p` subprocess | Agent SDK credit | Only via a Pi extension that shells out to it |
## Options
### Option A -- Status quo (Pi + ANTHROPIC_API_KEY)
Keep everything. No credit access. Pay full API rates personal and prod. Zero eng cost.
### Option B -- Install a Pi extension that adds the SDK as a provider ⭐ recommended starting point
Pi's extension system already has two community-maintained packages that register a Claude-Agent-SDK-backed provider:
- **[`claude-agent-sdk-pi`](https://github.com/prateekmedia/claude-agent-sdk-pi)** (96★, TS). Registers provider `claude-agent-sdk` with models like `claude-agent-sdk/claude-opus-4-5`. Routes LLM calls through `@anthropic-ai/claude-agent-sdk` directly. Pi keeps executing tools natively; the SDK only proposes. Loom's registered tools get exposed to the SDK via in-process MCP as `mcp__custom-tools__`. Auth via Claude Code login (Pro/Max) or `ANTHROPIC_API_KEY`. Install: `pi install npm:claude-agent-sdk-pi`.
- **[`pi-claude-cli`](https://github.com/rchern/pi-claude-cli)** (59★, v0.3.1 March 2026). Shells out to `claude -p` per request via stream-json wire protocol; follow-up turns use `--resume`. Tools via schema-only MCP server with a break-early pattern. Uses whatever auth `claude` on PATH already has.
**Cost: probably ~1 hour to install one and verify it works with Loom end-to-end. Zero Loom code changes** if Pi's provider-extension surface lets the existing tool/MCP wiring pass through cleanly (which it should -- these run at the provider layer, below the tool registration layer).
Caveats:
- Both are third-party community extensions. Sustainability risk if maintainers move on. Worth picking one and pinning a version.
- Whether `claude-agent-sdk-pi` with Pro/Max OAuth lands on the new Agent SDK credit bucket vs the interactive-subscription bucket is unverified in the extension's README. The SDK library is the credit-bearing path per Anthropic, but the empirical test happens post-install. `pi-claude-cli`'s `claude -p` path is unambiguously credit-eligible.
- Galaxy MCP via `~/.pi/agent/mcp.json` should still attach (MCP wiring is at the Pi layer, not the provider layer) -- needs spike verification.
- This is a personal/dev-machine answer. Shipping either extension to Galaxy end users runs into the third-party-claude.ai-login policy and needs the partnership conversation in Option E.
### Option C -- Bake extension support into Loom config / docs
If Option B's spike works, the next step is making it easy for other Loom devs to opt in: document `pi install` + `/model` in the Loom README, optionally have Loom's CLI detect when one of the extensions is installed and surface its models in `--list-models`. Small.
### Option D -- Replace Pi with claude-agent-sdk directly
Big lift. Rewrite the brain's LLM client around `@anthropic-ai/claude-agent-sdk` (Loom's brain is TS). Gain: cleaner SDK semantics, hooks API, native MCP wiring via `mcpServers` option, no community-extension dependency, Anthropic-supported runtime. Lose: Pi's multi-provider story (SDK is Anthropic-only), Pi's tool registration shape, and whatever Pi-specific features (notebook integration, eval matrix wiring, team_dispatch) we depend on. Months, not days. Only makes sense if (a) Option B's spike fails for fundamental reasons or (b) we want to leave Pi for unrelated reasons.
### Option E -- Partner approval for subscriber-auth in Loom-as-a-service
Separate track. Galaxy users bring their own Claude subscriptions; Loom-as-a-service authenticates each user against their sub. Massive upside (cheaper Loom for the Galaxy community, especially usegalaxy.org users), gated entirely on Anthropic partnership conversation per the explicit third-party policy quote. Do not block any of A-D on it. Should be its own issue.
### Recommendation
**Start with Option B as a 1-hour spike.** Install `claude-agent-sdk-pi` against a Max subscription, point Loom at it via `/model`, run one end-to-end Galaxy analysis, and verify:
1. MCP attaches and Galaxy tools work.
2. Loom's registered tools (`registerPlanTools` etc.) dispatch correctly through the SDK-pi MCP shim.
3. Usage actually charges to Agent SDK credit, not subscription quota or API rates (post-June-15 -- check the Anthropic Console usage view).
If all three pass, personal-use Loom on credit is a config-and-docs change. If gaps surface, decide between patching the extension upstream, writing a tiny Loom-side adapter, or escalating to Option D.
## Proposed next steps
- [ ] (Option B spike) Install `claude-agent-sdk-pi`. One end-to-end Galaxy analysis. Verify the three points above. ~1 hour.
- [ ] Extend `extensions/loom/teams/tool.ts:72-78`'s `msg.usage` accumulation to the single-agent path so we get per-session token/cost totals dumped to a file. Independently useful regardless of which option lands. ~30 min.
- [ ] Confirm what `ANTHROPIC_OAUTH_TOKEN` (`bin/loom.js:206`) actually authenticates against in Pi-AI (`packages/ai/src/utils/oauth/`). Probably Pi's own Claude Pro/Max OAuth, hitting the interactive-subscription bucket -- but worth verifying so we don't accidentally tell users that path is credit-eligible. ~30 min.
- [ ] If B spike passes: doc PR (Option C) -- README section on running Loom against an Anthropic subscription via the Pi extension.
- [ ] Separate issue tracking partnership conversation with Anthropic for multi-user Loom-as-a-service (Option E).
## Out of scope
- Implementation of any of the above
- Changes to how `usegalaxy.org` bills end users
- Changes to GalaxyMCP itself
- Replacing Pi for reasons unrelated to billing
## Open questions
- **Billing-bucket verification.** `claude-agent-sdk-pi` with Pro/Max OAuth -- does it land on the Agent SDK credit bucket or the interactive-subscription bucket? Has to be answered empirically post-install.
- **MCP pass-through with the extension.** Does `~/.pi/agent/mcp.json` (Galaxy MCP + BRC Analytics) attach normally when a third-party provider extension is in use, or does the SDK-pi extension override it? Spike target.
- **Tool dispatch through the SDK-pi MCP shim.** Loom registers ~dozen custom tools into Pi. The extension claims to expose these as `mcp__custom-tools__`. Need to confirm Loom's `team_dispatch`, `skills_fetch`, `galaxy_invocation_*`, and `gtn_*` tools all show up and dispatch correctly.
- **What happens mid-session when credit refreshes?** Continuity, reset, or token re-issue? Not in any of the docs I could find.
- **Community-extension sustainability.** Both extensions are maintained by individual contributors. What's our fallback if `claude-agent-sdk-pi` goes stale? (Probably "vendor a fork" -- but worth saying out loud.)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with package.json, extensions/loom/index.ts, and bin/loom.js to understand Pi provider installation, tool registration, and authentication. Install claude-agent-sdk-pi, select it with /model, and run one Galaxy analysis. Done means MCP and Loom tools work end to end and the Anthropic Console confirms usage charges to Agent SDK credit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, backend
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100