stacklok / stacklok/modelith

Skill: author property tests from a model's invariants and actions

Open Beginner friendly
#5 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
32
Forks
5
PR merge metrics
No merged PRs in 30d

Description

Summary

Add a Claude Code plugin skill that reads a *.modelith.yaml model and drafts property-based tests for its invariants and for the action.preserves links, in the user's language, with the human in the loop. The generated tests land in the user's repo and are owned by the user from line one — the skill never claims regeneration ownership.

This is the "wire testing in from the model" idea in its honest form: an invariant is a universally-quantified statement ("for all Projects, the Project has an owner"), which is exactly the shape of a property-based-testing property ("for all inputs x, the property holds"). The model captures the what; the test proves the code does. modelith sits one step upstream of where the property test lives.

Why a skill, not a generate subcommand

invariant.statement, entity.definition, and attribute.derivation are free-text prose — modelith is deliberately a communication artifact, not a spec language with an expression grammar. Turning "a Project has exactly one owner" into a working generator + assertion is a judgement task, not a deterministic compile. That belongs in an LLM skill, not the binary. It also keeps the binary lean (no language-backend matrix, no golden-fixture burden, no generated-code drift lifecycle).

Scope

  • New skill under plugin/skills/ (working name domain-model-test-author), a sibling to the existing model-context skill.
  • Input: a model file (+ the entity structure it references).
  • Output: property-test drafts, one per invariant.id, plus a "calls the action, asserts the invariant still holds" test per action.preserves entry.
  • Language-aware suggestions: testing/quick or pgregory.net/rapid (Go), fast-check (TS), hypothesis (Python). For this repo's own dogfooding, testing/quick is stdlib and adds zero deps.
  • Human-in-the-loop: the skill drafts; the engineer reviews, edits, and owns the file. No "DO NOT EDIT" banner, no regeneration contract.

Smallest thing that proves or kills it

Hand-run the skill against examples/example.modelith.yaml for two invariants — e.g. at-least-one-owner and no-archive-with-enabled-policies — and judge whether it produces tests an engineer would actually keep. Cost: one SKILL.md and an afternoon. If the tests are good, the valuable tier is validated with zero binary risk. If they're mush, the whole concept is killed cheaply.

Relationship to the other issues

  • Emits a traceability convention (invariant-id → test) that #6 can check.
  • Should be built before #7; #7 is gated on whether this human-in-the-loop, skill-based model proves out.

Background

Descends from the "Tier 3" of a three-tier design discussion. A skeptic pass concluded this is the only tier whose value and home are both honest — the code-generation tiers were reshaped (see #6, #7). Consider recording the decision in audits/.

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the existing model-context skill under plugin/skills/ and the model structure referenced by examples/example.modelith.yaml. Hand-run the new skill against the two named invariants and action.preserves entries, then check that it produces reviewable, language-aware property-test drafts with invariant-to-test traceability and no regeneration-ownership instructions.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
testing-qa, tooling
Issue type
Feature
Difficulty
2/5
Estimated time
Half a day
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.