ContextLab / ContextLab/claude-skill-compounder
Deeper skill composition, and the last step of the loop: the forged skill actually used
- Dominant language
- Python
- Stars
- 1
- Forks
- 0
- Avg merge
- 1h 29m
- Merged PRs (30d)
- 1
Description
here's the gap, which i'm still not seeing us solve correctly: what i want to happen is:
1. some issue comes up repeatedly. this could be:
- a task that gets done repeatedly-- e.g., red-teaming using subagents to verify correctness, then re-running all tests and linters, verifying everything is green (or fixing if needed), then updating notes, then committing and opening a pull request. this happens every time a new task is done. so we can operationalize it as a single "skill" that proceeds deterministically. that way the user doesn't need to remind the model to do all of those things, and the model (and the user) doesn't forget a step or skip a step.
- a mistake that is repeated several times (e.g., the "naive" thing a fresh session tries is X, but then after failing a few times it realizes the task needs to be done using approach Y).
- one example: the built-in skill for working with github isn't connected properly. but each fresh session tries to use that skill, fails, then retries with "gh" commands. it means every time github interactions are attempted, it takes several extra rounds of trial and error. this compounds to real wasted effort.
- another example: launching a huge "fan out" orchestration effort with several waves of steps occuring in parallel, at huge expense, only to discover at the end that the base assumption was fundamentally flawed and the entire operation is wasted. this wastes everyone's time, effort, and money. instead, the base assumptions need to be checked and made 100% rock solid BEFORE major efforts are made. again, this can be built as a skill for starting work-- first put in a big effort making sure the base assumptions are perfect. THEN proceed with an MVP. THEN test rigorously, including making real calls and verifying the intended use cases are covered. THEN build it out to the full original spec, and with each new feature verify (a) nothing has broken and (b) that new feature is justified, well-researched, and necessary BEFORE major effort to build it is spent.
- another example: a toolbox or codebase is changed and the model forgets to update the documentation. this is a missed opportunity for a *deterministic* hook-- before pushing updates, force a subagent to update the documentation to make it current.
- in general, anything where the user has to remind the model several times (perhaps even for different tasks or projects!) to course correct means that a skill could be implemented.
2. a skill is built to "smooth" that frequent occurence. the reason for using a skill is that it enables a combination of plain language (allowing for generalizeable qualitative direction) AND deterministic scripts/code (preventing things from going off course, improving reliability, reducing overall compute costs). critically, skills must also be hooked into the existing claude code infrastructure so that they AUTOMATICALLY trigger as needed (in addition to being manually callable). using a skill cannot rely on the "hope" that an agent that is in the middle of a massive effort and near the end of its context window will happen to act on a quick reminder injected into its context as a prompt. reminders need to either happen at times when the model will actually listen to them, or they need to be FORCED by hooking into the deterministic architecture and running them automatically.
3. skills compound and compose. this is where the real power comes from:
- skills can call *other* skills as part of their operation. e.g., maybe we have a red-team skill that generates adversarial reviews, an update-documentation skill that examines the code base and compares it to the current documentation (updating as needed), and a push-to-github skill that commits/pushes to a fork/opens a pull request/verifies all tests go green (or fixes as needed). the push-to-github skill might call the red-team skill internally as part of its self-review, and/or it might call the update-documentation skill when code is changed to make sure that pull requests always include up-to-date documentation. and/or *all* of these could be combined into a single improve-package skill that calls *each* of these skills in sequence, and perhaps adds some additional machinery.
- skills can also be scoped for the project/user/general public. this, too, helps with compositionality. suppose we determine that each project really needs its own compile-documentation skill, because there are too many idiosyncracies across projects to fully generalize this skill. (note: i'm not saying this is necessarily true-- it's just a hypothetical example!) first of all, if we notice that we're remaking the same skill for every project, it could potentially lead to a create-compile-documentation-skill skill that is defined at the user level and is called by individual projects. and at an even higher level, the create-compile-documentation skill might in turn call a general create-skill skill that is defined at the public level.
4. when a new skill is created, several things happen:
- the user receives a clear and obvious notification (could be in the status bar OR in the main thread) showing that the forge is being fired up, and why.
- the notification (or another object) then runs in the background as the skill is built and tested
- when the tool is built and ready for use, the user receives a notification that the new skill is now avaiable (and skills hot reload so that it can be used).
- then, since skills are built AS NEEDED, this implies that the skill also needs to be *used*. so the user should also get a notice that the skill is now being used to solve the problem it was created for.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by inventorying the existing skill, hook, notification, hot-reload, and test entry points in the repository, then run the current test suite; done requires an agreed scope and design for composition, automatic use, background building, notifications, and hot reload.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100