Optional --emit-receipts flag: portable, self-verifying result records (converter + PR offered)
- Dominant language
- TypeScript
- Stars
- 711
- Forks
- 46
- PR merge metrics
- No merged PRs in 30d
Description
Hi — I maintain [Driftproof](https://driftproofhq.com) ([repo](https://github.com/driftproofhq/driftproof)), a runner that measures agent-skill eval results across model releases and emits signed, dated **receipts** — an open JSON format with a published schema ([spec](https://driftproofhq.com/spec/receipt.schema.json), [interop guide](https://driftproofhq.com/interop.html)).
skillgrade's trial model maps onto the format surprisingly well: per-trial rewards are genuine repeated observations, so they convert into a receipt's `samples[]` with an honest cross-trial band (mean ± stddev per task) — which is more statistical structure than most eval outputs carry. I've already built and shipped the consuming side:
```
driftproof import --from skillgrade
```
converts results into a schema-valid receipt today (field mapping documented [here](https://driftproofhq.com/interop.md)). Two things the conversion is deliberately honest about: imported receipts are `verification_level: "DECLARED"` with no fabricated hashes (my tool didn't run your trials), and since skillgrade has no with/without-baseline mode, the receipt's comparison fields are `null` — never a fabricated zero-baseline delta. One honest caveat: the persisted results file has no frozen schema, so the converter's contract is a checked-in fixture authored from your documented semantics — if a real results file differs anywhere, tell me and I'll adjust.
**The proposal** — an optional `--emit-receipts` flag (default off, zero behavior change otherwise) that writes a receipt per skill into the results directory. Emitting natively would let you go beyond what my converter can honestly claim: you hold the transcripts and grader outputs at run time, so you could hash them into the receipt and emit at the full `TESTED` evidence level — making a skillgrade run portable, self-verifying, and regression-diffable across agent/model versions by any receipt-aware tool (which pairs naturally with your `--regression` mode).
What's in it for your users: a stable, documented interchange output for CI dashboards and badges that doesn't couple consumers to the results directory layout.
**I'm happy to write the PR** (TypeScript, additive, flag-gated, with tests) if you're open to it — or if you'd rather I just pin my converter to a stable subset of your persisted results, telling me which fields you consider contract would be enough. No pressure either way; the converter already works, this would just make it unnecessary.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the existing results writer and the --regression path, then review how transcripts and grader outputs are retained during a run. The completed work should add an optional, default-off --emit-receipts flag that writes one schema-valid receipt per skill, with evidence hashes where supported, without changing existing output behavior. Add tests covering the flag and receipt output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- cli, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100