memory-ingest: transcripts get no source isolation / repo-scope / trust enforcement → land in default (federated) or the code source; dream then mines them
- Dominant language
- TypeScript
- Stars
- 133k
- Forks
- 19.9k
- Avg merge
- 18h 46m
- Merged PRs (30d)
- 26
Description
## Summary
`gstack-memory-ingest` ingests coding-agent transcripts with **no source isolation, no repo/workspace scope, and no trust enforcement** — contrary to what `setup-gbrain` documents. The practical result: transcript pages land in the `default` source (often *federated*) or in the current repo's code source, and gbrain's `dream`/`extract` cycle then mines every one of them as if it were curated knowledge. On a real corpus this smears unrelated context across the brain and produced a **~21M-token blowup** the first time `dream` ran over the ingested transcripts (had to delete everything afterward).
Versions: gstack `1.58.5.0`, gbrain `0.42.42.0`.
## What the docs promise
- `setup-gbrain/SKILL.md:1456-1470` — transcript ingest default scope is **"current repo only, last 90 days"**, and **"Per-repo trust policies still apply."**
- `setup-gbrain/SKILL.md:1330` — `read-only` repos are skipped (but "enforced by the **future** auto-import hook … **not here**").
- `setup-gbrain/memory.md:35-37` — **"Repos under a `deny` trust policy … are skipped — neither code nor transcripts from those repos ingest."**
## What the code actually does
1. **Global walk, no repo/workspace filter.** `walkClaudeCodeProjects` / `walkCodexSessions` (`bin/gstack-memory-ingest.ts:362-423`) enumerate **all** of `~/.claude/projects/*` and `~/.codex/sessions/*`. `preparePages` (`:1130-1191`) filters only on mtime-dedup, optional secret-scan, and "unattributed" (no `cwd`/`git_remote`). There is no current-repo scope and no date floor beyond the 90-day window (`--all-history` removes even that).
2. **No trust enforcement.** The binary never reads `~/.gstack/gbrain-repo-policy.json`. `gstack-gbrain-repo-policy`'s own header says the tier is "**enforced at the caller level; this binary just stores the decision**." Nothing in `gstack-memory-ingest` or `gstack-gbrain-sync` consults it, so `deny`/`read-only` repos are **not** skipped for transcripts.
3. **No source isolation.** Ingest calls `gbrain import --no-embed --json` (`bin/gstack-memory-ingest.ts:1419`) with **no `--source-id`**. gbrain's resolver only auto-adopts a source when exactly one non-default source exists (`src/commands/import.ts:131-146`, `sole_non_default` tier). With multiple registered sources, transcripts route to **`default`** (which is commonly *federated*) — or, when `GBRAIN_SOURCE`/`.gbrain-source` is active, to the **current code source**. Never to a dedicated, isolated transcript source.
4. **The cycle then mines them.** `dream`/`extract` run per-source and do a full walk with **no `type` filter** (`src/core/cycle/extract-facts.ts:184` → `engine.getAllSlugs()`; sourceId scoping at `:98`). Every `type: transcript` page is treated as fact/take source material → one LLM call per page/chunk → the token blowup, plus low-quality facts smeared into the brain.
Note: `gstack-config set transcript_ingest_mode off` does **not** stop this at sync time — `transcript_ingest_mode` is referenced only in the `setup-gbrain` skill files, never read by `bin/gstack-gbrain-sync.ts` or `bin/gstack-memory-ingest.ts`. The only effective off-switch is `gstack-gbrain-sync --no-memory` / `--code-only`.
## Repro (no writes)
```sh
# 1. Brain has >1 registered non-default source (typical multi-project setup).
gbrain sources list
# 2. A bare sync runs the global transcript walk and imports with no --source-id:
gstack-gbrain-sync --incremental # -> runMemoryIngest -> gbrain import (no --source-id)
# 3. Resulting transcript pages carry source_id = 'default' (or the CWD code source),
# not an isolated transcript source. dream/extract over that source then mine them.
```
## Suggested fixes
1. **Route transcripts to a dedicated `--source-id`** (default a **non-federated** source, e.g. `transcripts-`), never `default`/the code source.
2. **Enforce repo-trust + an actual scope** *before* the walk: honor `gbrain-repo-policy.json` (skip `deny`/`read-only`), default to current-repo, and support a hard date floor — matching what `setup-gbrain` already promises.
3. **Let the cycle exclude `type: transcript`** (or make transcript ingest opt-in to mining), so `dream`/`extract` don't burn tokens treating raw transcripts as curated knowledge.
4. Make `transcript_ingest_mode=off` actually gate the sync memory stage (today it's docs-only).
## Related
Pairs with #2104 (staging inside `GSTACK_HOME`'s `gitignore=*` repo → git-aware import sees 0 files). Together: when the staging bug is fixed, transcripts *will* flow — and without this issue's scoping/isolation they'll flow into the wrong source and get mined.
Contributor guide
Research direction
Start with bin/gstack-memory-ingest.ts and bin/gstack-gbrain-sync.ts, then trace gbrain import in src/commands/import.ts and cycle extraction in src/core/cycle/extract-facts.ts. Reproduce the multi-source sync described in the issue and inspect how source, repository scope, trust policy, and transcript mode are handled. Done means transcript ingestion is scoped, isolated, trust-aware, and excluded or gated from dream/extract as intended.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- data, security, tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100