awslabs / awslabs/aidlc-workflows

feat: warm re-scan / no re-interrogation on iteration (reuse CodeKB + prior answers)

Open
#1,122 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
4.6k
Forks
827
Avg merge
1d 21h
Merged PRs (30d)
165

Description

### Problem

On any iteration over an existing workspace/intent, AI-DLC re-runs a cold Initialization scan and re-interrogates the user with questions they already answered. A customer reported a 45-minute re-scan that re-asked answered questions for what was ultimately an 11-line fix. This is the "reuse prior workspace understanding" half of the last-30% iteration gap.

### What already exists (verified against source)

The code-understanding side is largely built and should be reused, not rebuilt:

- Durable space-level CodeKB store `aidlc/spaces//codekb//` (`core/tools/aidlc-lib.ts` `codekbDir`).
- Git-tree-hash freshness fingerprint (`codekbScopeFingerprint` / `codekbSourceFingerprint`) and a `scope_version:1` freshness block in `reverse-engineering-timestamp.md`.
- A `CURRENT/STALE/...` verdict (`codekb-scope-diff`) + a reuse gate in the reverse-engineering stage (Reuse / Full rescan / Focused scan), so the `--rescan full` escape hatch effectively exists already as "Full rescan".
- `propagateStageInvalidation` (`core/tools/aidlc-validity.ts`) already marks downstream artifacts `stale`/`needs-revalidation` **without resetting completed stages** (#716 mechanism).

### The two real gaps

1. **Per-file delta, not store-wide.** The freshness verdict today is whole-scope `CURRENT/STALE` — one changed file flips the entire store STALE with no changed-file set, so there is no "re-scan only what changed". Add a delta computation (git diff of the store's recorded generation vs HEAD, on the existing fingerprint seam) that returns the changed-file set, and make the reverse-engineering developer brief accept it.
2. **Answered questions are not persisted as reusable Q→A pairs at all.** `QUESTION_ANSWERED` events are write-only audit rows read back only for within-attempt authorization — never to pre-fill a later run. A durable, provenance-stamped answer store + a pre-fill reader (invoked before a stage's question fences) is a from-scratch component. Surface "reused N prior answers"; a changed question or changed code must re-ask rather than silently reuse.

### Acceptance

- A second workflow over the same repo/intent completes Initialization by scanning only changed files and reports the cache hit + delta.
- Questions answered in the first run are not re-asked; the run reports how many prior answers were reused.
- `--rescan full` bypasses the cache and re-scans everything.

### Notes

- Foundational: unblocks the `amend` scope and batch-ingestion work.
- Related: #716 (stale-result propagation, the reuse target), #414 (incremental/versioned artifact history — adjacent), #1094 (clarifying-question ordering — adjacent to the re-ask logic). Roadmap Goal 5 (cyclic flows) / Goal 6.
- Author in `core/`/`harness/`; keep `dist/` generated; add tests at the appropriate tiers.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the [project license](https://github.com/awslabs/aidlc-workflows/blob/main/LICENSE).

Contributor guide

Open the contributing guide

Research direction

Start in core/tools/aidlc-lib.ts at codekbDir and the codekbScopeFingerprint/codekbSourceFingerprint seam, then trace the reverse-engineering reuse gate and core/tools/aidlc-validity.ts. Follow QUESTION_ANSWERED handling in the harness and add tests at the appropriate tiers. Done means changed-file deltas and provenance-checked answer reuse are reported, full rescan bypasses reuse, and the stated acceptance scenarios pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, typescript
Domain
developer-experience, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.