galaxyproject / galaxyproject/loom
Right-size always-on system prompt + tool surface for smaller open models
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
## Symptom
Running Loom in non-interactive mode (`-p`) against gpt-oss-120b (via a local litellm relay) consistently hangs on the first turn -- the model produces only a `reasoning_content` block and never reaches a `content` or `tool_use` response. The session ends with a clean `notebook_snapshot` and zero work done.
Same model + same prompt + a tiny ad-hoc tool surface (single tool, no system prompt extras) returns clean tool calls in <1s. Same model on a focused Loom prompt with `--no-skills --no-context-files --thinking off` works end-to-end (real tool calls, real BRC MCP queries, notebook written).
So the model is capable. The failure mode is the *combined weight* of Loom's always-on context: skill-router blurb in `extensions/loom/context.ts:561-609`, MCP tool lists (galaxy-mcp's ~20 + brc-analytics' 12 + pi built-ins + web access + Loom extension tools), plus AGENTS.md / CLAUDE.md ambient injection. At full surface, smaller open models exhaust themselves on reasoning before producing output.
## Reproducer
```bash
mkdir /tmp/loom-tier0-repro && cd /tmp/loom-tier0-repro
LOOM_FRESH_SESSION=1 node /path/to/loom/bin/loom.js -p \
--provider litellm --model gpt-oss-120b \
"Use the brc_analytics_search_organisms tool to find Plasmodium falciparum and report the result."
# Hangs on first turn. Activity log shows assistant message with only 'thinking' part, no text/tool_use.
# Same prompt with reduced surface succeeds:
LOOM_FRESH_SESSION=1 node /path/to/loom/bin/loom.js -p \
--provider litellm --model gpt-oss-120b \
--no-skills --no-context-files --thinking off \
"Use the brc_analytics_search_organisms tool to find Plasmodium falciparum and report the result."
# Calls the tool, gets NCBI taxonomy 5833 + 27 assemblies, writes a markdown table response.
```
This isn't gpt-oss-120b-specific -- it's a class effect. Llama-4-Maverick / Sonnet / Opus would likely shrug it off. But "smaller open models hit the wall sooner" is exactly the audience Loom is trying to support for air-gapped / cost-sensitive / Galaxy-community deployments.
## Proposed approach
Treat always-on context as a **budget**, not a bag of features that grow as we ship. Three concrete steps:
1. **Audit current Tier-0 size.** Add a one-time startup diagnostic that prints the assembled system prompt + tool-list size in tokens (per provider's tokenizer where possible, char-count approximation otherwise). Logs to stderr behind `--verbose`, doesn't change UX. Lets us measure.
2. **Define a Tier-0 budget per model class.** Probably ~5K tokens for "small-open" models, ~15K for hosted frontier. Stored as model metadata in `~/.pi/agent/models.json` or shipped in Loom's defaults. When the assembled Tier-0 exceeds the configured budget, log a warning at startup ("system prompt 12.3K tokens; configured budget 5K -- consider reducing skills router or other always-on sections").
3. **Make `--print` mode default to a leaner Tier-0.** In non-interactive runs the user almost never benefits from the welcome greeting, the skill router (they're invoking a single shot, not browsing skills), or ambient CLAUDE.md context. Today they have to pass three flags. Reasonable defaults for `--print`:
- skip the session-start welcome (already done if `LOOM_FRESH_SESSION=1`)
- skip skill-router prose (skill *fetching* still works on demand)
- keep CLAUDE.md / AGENTS.md (low-cost, often useful)
Make the existing flags overrides rather than required for bare survival.
## Why this matters now
Loom's always-on context grows naturally as features ship -- new typed notebook blocks add summary lines to the digest, new MCP registrations add tool descriptions, new grounding sources add router blurbs. Each addition individually passes the "+1 line is fine" test; the aggregate slowly excludes smaller open models. Catching the budget question now is cheap; catching it after several more features compound is a refactor.
This is also the empirical motivation for context-management discipline as Loom rides on Galaxy's full surface area (~8K tools, IWC workflows, GTN, BRC organism / assembly / workflow catalog, etc.). The substrate is what makes Loom interesting; without budgeting the surface that actually hits the model, the substrate becomes the failure mode.
## Open questions
- Does pi expose a hook to introspect the assembled system prompt? If not, the audit diagnostic might need a cooperative addition upstream.
- For the `--print` defaults change: should the lean profile be opt-out (`--print` defaults to lean, `--print --full` to keep current behavior), or opt-in via a separate flag (`--print-lean`)? Opt-out is safer if pre-existing scripts rely on the current behavior; opt-in is more discoverable.
- Is there a clean way to express "fetch this skill on demand but don't put its router blurb in Tier-0" -- so the skill is reachable via `skills_fetch` without paying the prompt cost? That's the architectural question behind this whole thread.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.