1jehuang / 1jehuang/jcode

Add closed-loop skill distillation for reusable coding workflows

Open
#61 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement priority: medium triage: needs-decision
Dominant language
Rust
Stars
19.9k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
30

Description

Feature request

jcode already has a strong foundation for long-running coding work: session memory, semantic retrieval, skills, multi-session workflows, and self-improvement-oriented tooling.

A useful next step would be a closed-loop skill distillation flow: when the agent completes a non-trivial task, it can turn the learned workflow into a reusable SKILL.md, ask the user to approve it, and later retrieve or patch that skill when similar work appears again.

Hermes Agent has a similar idea where skills act as procedural memory: the agent can create or update skills after complex tasks, failed attempts that eventually succeed, user corrections, or discovered workflows. jcode seems especially well positioned for this because it already has skills plus semantic memory/retrieval.

Proposed behavior

After a session, task, or autonomous loop finishes, jcode could run a lightweight "skill distillation" pass.

It should detect whether the task is worth learning from, for example:

  • many tool calls
  • repeated failed attempts before success
  • user corrected the agent
  • a project-specific workflow was discovered
  • tests, build, lint, auth, packaging, or release steps revealed a non-obvious fix

If useful, jcode generates a candidate skill with:

  • name
  • description
  • when to use
  • procedure
  • pitfalls
  • verification steps
  • relevant commands
  • optional references, templates, or helper scripts

Before saving anything, jcode should ask the user what to do:

  • save as a new skill
  • patch an existing skill
  • discard
  • save project-locally only
  • save globally

Storage and retrieval

Suggested storage:

  • project-local skills: .jcode/skills/
  • global skills: ~/.jcode/skills/
  • optional read-only external skill directories later, for compatibility with shared SKILL.md libraries

Once saved, the skill should be reusable through existing jcode mechanisms:

  • semantic skill retrieval
  • slash command / manual activation
  • explicit skill search or selection

Skill improvement loop

If a skill is used later and fails, jcode could propose a patch instead of silently moving on.

Examples:

  • user corrects a command from the skill
  • tests fail and the agent discovers an extra required step
  • a provider/auth workflow changes
  • a generated file or snapshot update procedure was incomplete

In those cases, jcode can propose a focused update to the relevant skill, again requiring user approval.

Why this fits jcode

Coding agents often waste tokens rediscovering the same repo-specific workflows:

  • how this repo runs tests
  • how migrations are generated
  • how snapshots are updated
  • how provider auth is debugged
  • how release artifacts are built
  • how generated bindings are refreshed
  • how jcode itself should be modified safely

This feature would let jcode compound useful procedural knowledge over time without model fine-tuning. The learning stays external, auditable, versionable, cheap, and reversible.

Safety considerations

Agent-created skills should not silently become trusted instructions.

Suggested safeguards:

  • require user approval before saving or patching skills
  • distinguish global vs project-local skills clearly
  • mark agent-created skills as unreviewed until accepted
  • never save secrets, tokens, private URLs, or raw credentials
  • keep an audit log for create / patch / delete actions
  • provide commands to list, inspect, diff, disable, and reset skills
  • scan skill content for prompt injection, credential leakage, and destructive shell commands

Possible MVP

A small MVP could be:

  1. Add an end-of-task skill distillation prompt.
  2. Generate a candidate SKILL.md.
  3. Show a diff/preview to the user.
  4. Save only after explicit approval.
  5. Store project-local skills under .jcode/skills/.
  6. Make saved skills available through existing skill retrieval.

Global skills, external read-only directories, skill scoring, pruning, and hub/import support can come later.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating jcode’s existing skill storage, semantic retrieval, and end-of-task flow, then review the proposed .jcode/skills/ and SKILL.md formats. A first MVP is done when an approved candidate skill can be previewed, saved project-locally, and retrieved through the existing skill mechanisms without saving secrets.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai, cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.