ContextLab / ContextLab/claude-skill-compounder

EPIC: Three tiers of compounding: note, reminder, skill

Open
#31 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1
Forks
0
Avg merge
1h 29m
Merged PRs (30d)
1

Description

From the audit in `notes/2026-09-02-audit-and-replan.md` (five parallel read-only agents,
2026-09-02). Synthesis and plan, quoted from that note.

## Synthesis

> The system has exactly one output path, the forge, and that path costs a median 3.3 hours
> and 8 agents per skill while the cheapest forms of compounding (a line in CLAUDE.md, a
> reminder injected when a matching prompt or command appears) have no mechanism at all.
> SKILL.md line 44 says "write a note or update CLAUDE.md" and names no path, no CLI, no
> ledger row. It has been taken zero times in ten days. The insight queue has 57 candidates
> in and 0 out. Two paid CANDIDATE verdicts produced nothing. The one artifact that both
> accumulates and fires automatically is the repeat store, and it is the smallest, dumbest,
> most deterministic piece: a content-addressed signature matched on PreToolUse. That is the
> shape the rest should copy.

> Second diagnosis: the measurement layer that would tell us whether any of this works is
> either dead (reminder-conversion counter cannot parse its own tally), inflated
> (skillreport credits the same uses to three forge rows), or never invoked (`skillforge
> verdict`, 0 rows in 807; 5 of 6 closed forges have no apply row). We cannot tune anything
> until it reports.

> Third: the forge protocol is 12 steps and 28 doctrine gates, and the audit could attribute
> observed catches to only four cheap pieces (parse gate, routing gate, round 1 plus one
> confirming round, non-fork reviewer). The orchestrator layer catches nothing and caused
> the 86-hour stuck forge. Stage E has found two meta findings ever. Rounds 3..N are ~60% of
> wall clock and the round record shows repairs creating new findings as often as closing
> them.

## Plan: three tiers of compounding, one promotion path, honest instruments

> **Tier 0, NOTE.** A dated line in a marker block of `./.claude/CLAUDE.md`,
> `~/.claude/CLAUDE.md`, or a project memory file. Seconds. Written by `skillnote` (new
> CLI), which also writes a `note` ledger row. The forge's "cheap branch" and every
> CANDIDATE verdict land here.

> **Tier 1, REMINDER.** A searchable store (`/reminders.jsonl`) of short prose keyed
> on keywords, paths, or command signatures; a UserPromptSubmit/PreToolUse hook injects
> matches as additionalContext. Generalises what repeat-gate already does for command
> signatures. Written by `skillnote --remind`, and by the insight queue when a candidate
> recurs.

> **Tier 2, SKILL.** SKILL.md plus optional scripts, forged by ONE builder and ONE cold
> reviewer, default two rounds, parse gate and routing gate kept, orchestrator layer dropped
> for two-round forges, judge folded into the brief, repro optional. Target: under 30
> minutes for a narrow skill. Escalate rounds only on a converging record; hard cap.

> **Promotion:** queue -> note -> reminder -> skill, driven by recurrence counted the way
> the repeat store counts it. `skillinsight promote` moves a candidate up one tier.

> **Instruments:** fix the unary counter, dedupe skillreport, wire or remove `skillforge
> verdict`, index the lost verdict, add `skillforge doctor` and a TTL reaper for stuck
> forges.

## Status

Waves 1 through 4 are committed: `03773ad`, `7507a0b`, `15b3b28`, `c06eb6c`. The checklist
that stood here is replaced by what each issue actually reached, verified against the tree at
`385624f` on 2026-09-03. **Implemented** means the code is in and has tests. **Live-proven**
means it ran against a real install and produced an artifact somebody read. **Measured**
means a number exists that a later run can be compared against.

|issue|implemented|live-proven|measured|notes|
|-|-|-|-|-|
|#24 measurement fixes|yes|yes, `skillreport` computes on 115 real counters|yes, REUSE 5 of 6 counted once per skill|closed; `verdict` wiring unexercised, moved to #34|
|#25 doctor and reaper|yes, 41 tests|yes, `skillforge doctor` 8 pass 0 fail on this install|n/a|closed; the reaper has never fired on a real stuck forge|
|#26 installer doctrine stanza|yes|yes, block at `~/.claude/CLAUDE.md:96-144`, rerun reports already current|n/a|closed|
|#29 trivia|yes|yes, both figures derived rather than restated|n/a|closed|
|#30 conversion baseline|n/a|n/a|yes, 10.5% over 1456 transcripts|**open**; sweep is not yet a script, and the next one must count the tiers separately|
|#20 `skillnote` (T0)|yes, 73 tests|yes, 18 `note` rows, 4 dated lines in `.claude/CLAUDE.md`|yes, memory read-back 3/3 via the index, 0/3 without|closed|
|#21 reminder store (T1)|yes, 73 tests|yes, 16 delivery rows, both arms 3/3 on 2.1.259|yes, 49 to 66 ms per event on 500 rows|closed; prune and bounds moved to #33|
|#22 forge diet (T2)|yes|**no**|**no**|closed as built; no forge has run under it, see #34|
|#23 promotion path|yes|yes, queue drained from 57 to 0 pending|yes, 46 declined, 12 promoted|closed|
|#27 repeat-gate|yes, refusal arm default off|yes, 10 threshold signatures denied 0|yes, via `--eligible-of`|closed; reopens if a non-allowlisted signature reaches threshold|
|#28 doc-gate, skillcontrib, skillrepeat|yes, all three fixed, none retired|partial: the three doc-gate refusals predate the fix|yes|closed|
|#8 PreCompact capture|yes, 47 tests|**no**, 0 rows with `source:"precompact"` in the queue|yes, payload and cost on 2.1.259|closed; budget and `custom_instructions` gaps moved to #32|
|#19 composition and the applied skill|partial|no|no|**open**, retitled and narrowed|

## Blocking everything above

**CI has been red on `main` since 2026-08-26**, on both matrix legs. #35, P0. A tracker that
closes issues against a red suite is asserting things it cannot check.

## What the user asked for

> i'm a little surprised that the skills take SO long to build-- i thought a skill was
> primarily a markdown file, plus some useful scripts? skills can be relatively simple and
> narrowly scoped, as long as they are useful. compounding knowledge can also be as simple
> as making a note in CLAUDE.md (the local or global version), or building a searchable
> reminder document that gets automatically injected into context at the appropriate times.

That is built. What it has not yet done is produce a skill under the new budget and show it
being used on the problem that caused it.

Contributor guide

Open the contributing guide

Research direction

Start with notes/2026-09-02-audit-and-replan.md and the wave status at tree 385624f, then inspect the open follow-up issues named in the epic, especially #30, #19, #32, #33, #34, and blocking #35. This is complete only when the three-tier plan is implemented within its stated budget, produces a skill, and shows that skill being used on the originating problem.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
developer-experience, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.