Dream: check unchanged synthesis completion before paid triage; separate screening deferral from synthesis backlog
- Dominant language
- TypeScript
- Stars
- 30k
- Forks
- 4.5k
- Avg merge
- 17h 59m
- Merged PRs (30d)
- 56
Description
## Problem
Dream synthesis checks whether a transcript's current content is already synthesized **after** relevance triage. This makes `not yet triaged` / screening deferral an unreliable proxy for synthesis backlog, and can spend paid triage calls on unchanged transcripts whose synthesis is already complete.
This is a pipeline-ordering and reporting problem, not just a label change.
## Code evidence
Inspected upstream master at `668b9bac302705f3bca0ae4792a49fab0a79a74e`:
- [`src/core/cycle/synthesize.ts`](https://github.com/garrytan/gbrain/blob/668b9bac302705f3bca0ae4792a49fab0a79a74e/src/core/cycle/synthesize.ts#L490-L560) runs triage, computes `worthProcessing`, and reports time-budget deferrals as `not yet triaged`. The dry-run return also precedes the completion checks.
- [Lines 637–638](https://github.com/garrytan/gbrain/blob/668b9bac302705f3bca0ae4792a49fab0a79a74e/src/core/cycle/synthesize.ts#L637-L638) load source-scoped completed synthesis keys only later.
- [Lines 684–721](https://github.com/garrytan/gbrain/blob/668b9bac302705f3bca0ae4792a49fab0a79a74e/src/core/cycle/synthesize.ts#L684-L721) already implement legacy and synth-v2 completion checks, including complete chunk sets, but inside the post-triage submission path.
A completed transcript with an absent triage cache entry can therefore consume a triage call, or remain counted as triage-deferred when the time budget expires, even though its current content does not need synthesis. This report is based on code inspection; the reproduction below is a proposed regression fixture, not a claimed executed test.
## Proposed smallest fix
Move/reuse the existing source-scoped, transcript-identity-and-content-hash completion checks before paid triage. Partition discovered transcripts into unchanged completed records and remaining candidates; run relevance triage only on the latter in the ordinary synthesis path.
Preserve existing legacy/v2 identity rules and full-chunk-set completion semantics. Do not introduce a new completion definition, broaden source matching, or treat partial/failed work as complete. Explicit retriage behavior should remain explicit rather than being silently changed.
Report synthesis state separately from screening disposition. At minimum distinguish:
- **Already synthesized; current content unchanged.** No paid screening or synthesis needed.
- **Awaiting first synthesis; screening deferred.** No prior completion established; screening has not finished.
- **Previously synthesized; current content hash differs.** Current content remains eligible for screening and synthesis.
- **Screened out as low relevance.** An actual screening verdict, not a timeout or provider failure.
- **Attempted but failed or incomplete.** Includes incomplete chunk sets; must remain retry-eligible under existing policy.
These dimensions can overlap (for example, changed content may also be screening-deferred), so expose separate synthesis-state and screening-disposition fields or document an unambiguous aggregation precedence. Also retain explicit eligible/pending, queued/running, budget-deferred, and degraded/unknown states where applicable rather than forcing everything into the five labels above.
`Not yet triaged` should describe screening state only. Do not present it as the count of transcripts requiring synthesis, and do not imply every candidate will pass relevance screening.
## Suggested regression fixture / acceptance criteria
1. Seed same-source legacy and v2 completed jobs for unchanged transcripts, including a fully completed chunk set, with no cached triage verdict. Ordinary synthesis and dry-run should classify them as already synthesized without calling the triage model for them.
2. Seed another source with the same path/hash: its completion must not suppress this source's work.
3. Change a previously completed transcript's content: it remains eligible and reports changed content rather than unchanged completion.
4. Seed partial chunks and failed/cancelled attempts: neither counts as complete; preserve retry behavior.
5. Exhaust the triage time budget with a mixed fixture: screening-deferred counts exclude unchanged completed records. Provider errors and unscored records must not become low-relevance rejections.
6. Verify structured output, human summary, and dry-run agree on the same candidate population, without double-counting overlapping states. An all-completed input should need no paid model calls.
7. Preserve relevance thresholds, rescue behavior, spending/submission caps, and the existing completion/idempotency contract. No broader backlog acceleration or new scheduler is needed.
## Outcome
An honest backlog and no paid screening of unchanged, already-completed work. A cosmetic rename alone would leave both the wasted screening and the misleading denominator intact.
Contributor guide
Research direction
Start in src/core/cycle/synthesize.ts by tracing triage, dry-run handling, and the existing legacy/v2 completion checks around the cited lines. Verify the proposed regression fixtures across unchanged, changed, partial, failed, source-mismatched, and budget-deferred transcripts; done means ordinary synthesis, dry-run, structured output, and summaries agree without paid triage for unchanged completed work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, backend, testing
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100