garrytan / garrytan/gstack

memory-ingest: transcripts get no source isolation / repo-scope / trust enforcement → land in default (federated) or the code source; dream then mines them

Open
#2,140 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133k
Forks
19.9k
Avg merge
18h 46m
Merged PRs (30d)
26

Description

## Summary

`gstack-memory-ingest` ingests coding-agent transcripts with **no source isolation, no repo/workspace scope, and no trust enforcement** — contrary to what `setup-gbrain` documents. The practical result: transcript pages land in the `default` source (often *federated*) or in the current repo's code source, and gbrain's `dream`/`extract` cycle then mines every one of them as if it were curated knowledge. On a real corpus this smears unrelated context across the brain and produced a **~21M-token blowup** the first time `dream` ran over the ingested transcripts (had to delete everything afterward).

Versions: gstack `1.58.5.0`, gbrain `0.42.42.0`.

## What the docs promise

- `setup-gbrain/SKILL.md:1456-1470` — transcript ingest default scope is **"current repo only, last 90 days"**, and **"Per-repo trust policies still apply."**
- `setup-gbrain/SKILL.md:1330` — `read-only` repos are skipped (but "enforced by the **future** auto-import hook … **not here**").
- `setup-gbrain/memory.md:35-37` — **"Repos under a `deny` trust policy … are skipped — neither code nor transcripts from those repos ingest."**

## What the code actually does

1. **Global walk, no repo/workspace filter.** `walkClaudeCodeProjects` / `walkCodexSessions` (`bin/gstack-memory-ingest.ts:362-423`) enumerate **all** of `~/.claude/projects/*` and `~/.codex/sessions/*`. `preparePages` (`:1130-1191`) filters only on mtime-dedup, optional secret-scan, and "unattributed" (no `cwd`/`git_remote`). There is no current-repo scope and no date floor beyond the 90-day window (`--all-history` removes even that).
2. **No trust enforcement.** The binary never reads `~/.gstack/gbrain-repo-policy.json`. `gstack-gbrain-repo-policy`'s own header says the tier is "**enforced at the caller level; this binary just stores the decision**." Nothing in `gstack-memory-ingest` or `gstack-gbrain-sync` consults it, so `deny`/`read-only` repos are **not** skipped for transcripts.
3. **No source isolation.** Ingest calls `gbrain import --no-embed --json` (`bin/gstack-memory-ingest.ts:1419`) with **no `--source-id`**. gbrain's resolver only auto-adopts a source when exactly one non-default source exists (`src/commands/import.ts:131-146`, `sole_non_default` tier). With multiple registered sources, transcripts route to **`default`** (which is commonly *federated*) — or, when `GBRAIN_SOURCE`/`.gbrain-source` is active, to the **current code source**. Never to a dedicated, isolated transcript source.
4. **The cycle then mines them.** `dream`/`extract` run per-source and do a full walk with **no `type` filter** (`src/core/cycle/extract-facts.ts:184` → `engine.getAllSlugs()`; sourceId scoping at `:98`). Every `type: transcript` page is treated as fact/take source material → one LLM call per page/chunk → the token blowup, plus low-quality facts smeared into the brain.

Note: `gstack-config set transcript_ingest_mode off` does **not** stop this at sync time — `transcript_ingest_mode` is referenced only in the `setup-gbrain` skill files, never read by `bin/gstack-gbrain-sync.ts` or `bin/gstack-memory-ingest.ts`. The only effective off-switch is `gstack-gbrain-sync --no-memory` / `--code-only`.

## Repro (no writes)

```sh
# 1. Brain has >1 registered non-default source (typical multi-project setup).
gbrain sources list
# 2. A bare sync runs the global transcript walk and imports with no --source-id:
gstack-gbrain-sync --incremental # -> runMemoryIngest -> gbrain import (no --source-id)
# 3. Resulting transcript pages carry source_id = 'default' (or the CWD code source),
# not an isolated transcript source. dream/extract over that source then mine them.
```

## Suggested fixes

1. **Route transcripts to a dedicated `--source-id`** (default a **non-federated** source, e.g. `transcripts-`), never `default`/the code source.
2. **Enforce repo-trust + an actual scope** *before* the walk: honor `gbrain-repo-policy.json` (skip `deny`/`read-only`), default to current-repo, and support a hard date floor — matching what `setup-gbrain` already promises.
3. **Let the cycle exclude `type: transcript`** (or make transcript ingest opt-in to mining), so `dream`/`extract` don't burn tokens treating raw transcripts as curated knowledge.
4. Make `transcript_ingest_mode=off` actually gate the sync memory stage (today it's docs-only).

## Related

Pairs with #2104 (staging inside `GSTACK_HOME`'s `gitignore=*` repo → git-aware import sees 0 files). Together: when the staging bug is fixed, transcripts *will* flow — and without this issue's scoping/isolation they'll flow into the wrong source and get mined.

Contributor guide

Open the contributing guide

Research direction

Start with bin/gstack-memory-ingest.ts and bin/gstack-gbrain-sync.ts, then trace gbrain import in src/commands/import.ts and cycle extraction in src/core/cycle/extract-facts.ts. Reproduce the multi-source sync described in the issue and inspect how source, repository scope, trust policy, and transcript mode are handled. Done means transcript ingestion is scoped, isolated, trust-aware, and excluded or gated from dream/extract as intended.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
data, security, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.