garrytan / garrytan/gbrain

Dream: check unchanged synthesis completion before paid triage; separate screening deferral from synthesis backlog

Open
#5,145 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
30k
Forks
4.5k
Avg merge
17h 59m
Merged PRs (30d)
56

Description

## Problem

Dream synthesis checks whether a transcript's current content is already synthesized **after** relevance triage. This makes `not yet triaged` / screening deferral an unreliable proxy for synthesis backlog, and can spend paid triage calls on unchanged transcripts whose synthesis is already complete.

This is a pipeline-ordering and reporting problem, not just a label change.

## Code evidence

Inspected upstream master at `668b9bac302705f3bca0ae4792a49fab0a79a74e`:

- [`src/core/cycle/synthesize.ts`](https://github.com/garrytan/gbrain/blob/668b9bac302705f3bca0ae4792a49fab0a79a74e/src/core/cycle/synthesize.ts#L490-L560) runs triage, computes `worthProcessing`, and reports time-budget deferrals as `not yet triaged`. The dry-run return also precedes the completion checks.
- [Lines 637–638](https://github.com/garrytan/gbrain/blob/668b9bac302705f3bca0ae4792a49fab0a79a74e/src/core/cycle/synthesize.ts#L637-L638) load source-scoped completed synthesis keys only later.
- [Lines 684–721](https://github.com/garrytan/gbrain/blob/668b9bac302705f3bca0ae4792a49fab0a79a74e/src/core/cycle/synthesize.ts#L684-L721) already implement legacy and synth-v2 completion checks, including complete chunk sets, but inside the post-triage submission path.

A completed transcript with an absent triage cache entry can therefore consume a triage call, or remain counted as triage-deferred when the time budget expires, even though its current content does not need synthesis. This report is based on code inspection; the reproduction below is a proposed regression fixture, not a claimed executed test.

## Proposed smallest fix

Move/reuse the existing source-scoped, transcript-identity-and-content-hash completion checks before paid triage. Partition discovered transcripts into unchanged completed records and remaining candidates; run relevance triage only on the latter in the ordinary synthesis path.

Preserve existing legacy/v2 identity rules and full-chunk-set completion semantics. Do not introduce a new completion definition, broaden source matching, or treat partial/failed work as complete. Explicit retriage behavior should remain explicit rather than being silently changed.

Report synthesis state separately from screening disposition. At minimum distinguish:

- **Already synthesized; current content unchanged.** No paid screening or synthesis needed.
- **Awaiting first synthesis; screening deferred.** No prior completion established; screening has not finished.
- **Previously synthesized; current content hash differs.** Current content remains eligible for screening and synthesis.
- **Screened out as low relevance.** An actual screening verdict, not a timeout or provider failure.
- **Attempted but failed or incomplete.** Includes incomplete chunk sets; must remain retry-eligible under existing policy.

These dimensions can overlap (for example, changed content may also be screening-deferred), so expose separate synthesis-state and screening-disposition fields or document an unambiguous aggregation precedence. Also retain explicit eligible/pending, queued/running, budget-deferred, and degraded/unknown states where applicable rather than forcing everything into the five labels above.

`Not yet triaged` should describe screening state only. Do not present it as the count of transcripts requiring synthesis, and do not imply every candidate will pass relevance screening.

## Suggested regression fixture / acceptance criteria

1. Seed same-source legacy and v2 completed jobs for unchanged transcripts, including a fully completed chunk set, with no cached triage verdict. Ordinary synthesis and dry-run should classify them as already synthesized without calling the triage model for them.
2. Seed another source with the same path/hash: its completion must not suppress this source's work.
3. Change a previously completed transcript's content: it remains eligible and reports changed content rather than unchanged completion.
4. Seed partial chunks and failed/cancelled attempts: neither counts as complete; preserve retry behavior.
5. Exhaust the triage time budget with a mixed fixture: screening-deferred counts exclude unchanged completed records. Provider errors and unscored records must not become low-relevance rejections.
6. Verify structured output, human summary, and dry-run agree on the same candidate population, without double-counting overlapping states. An all-completed input should need no paid model calls.
7. Preserve relevance thresholds, rescue behavior, spending/submission caps, and the existing completion/idempotency contract. No broader backlog acceleration or new scheduler is needed.

## Outcome

An honest backlog and no paid screening of unchanged, already-completed work. A cosmetic rename alone would leave both the wasted screening and the misleading denominator intact.

Contributor guide

Open the contributing guide

Research direction

Start in src/core/cycle/synthesize.ts by tracing triage, dry-run handling, and the existing legacy/v2 completion checks around the cited lines. Verify the proposed regression fixtures across unchanged, changed, partial, failed, source-mismatched, and budget-deferred transcripts; done means ordinary synthesis, dry-run, structured output, and summaries agree without paid triage for unchanged completed work.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend, testing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.