QuantEcon / QuantEcon/quantecon-plugins.mystmd

The AST conformance checker the contract promises does not exist

Open
#16 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
JavaScript
Stars
0
Forks
0
Avg merge
2h 52m
Merged PRs (30d)
7

Description

CONTRACT.md repeatedly delegates the invariants JSON Schema cannot express to a conformance checker, and that checker does not exist. CONTRACT.md:502 promises fixtures shipping with "a checker for the invariants JSON Schema cannot express — array-length equality, index correspondence, denormalised totals, deep-equality of a duplicated object". Nothing in scripts/ or tests/ implements it.

The effect is that several of the contract's load-bearing rules are enforced by nothing while both the prose and the schema descriptions tell an implementer they are machine-checked. A third-party wrapper runs npm run test:contract, sees green, and ships a fallback that disagrees with its own properties.

Found by an eleven-lens audit of the merged contract on 2026-09-03 (#14, squash f922dd7); each item was confirmed by a second agent told to refute it.

Invariant Stated at Enforced by
badges: item and span counts equal, field agreement, href both directions, strict badge/separator alternation CONTRACT.md:4627 nothing
delta-list: groups[i].items.length equals the listItem count in children[i] CONTRACT.md:5470 nothing
heatmap: fallback row count is rows.length + 1, and every row carries columns.length + 1 cells CONTRACT.md:2584 nothing
bar-list: "exactly one header row and it is row 0" schema/bar-list.json:5 nothing, and the same description contradicts itself two sentences earlier
no label, identifier or html_id on any node CONTRACT.md:180 two schemas of eight

Two further points sharpen the work. The stated reason for not encoding the arity rules in the schemas is false: stats (rowCountFollowsStats) and chips (chipCountFollowsItems) already encode the structurally identical coupling by enumerating a closed tuple per size, so heatmap and delta-list can do the same. And check 4 of the validator asserts only the four root properties, every one of which the schemas already enforce, so it currently adds no coverage at all.

The work. Add an AST conformance check to scripts/validate-contract.mjs that runs a per-primitive invariant function over every valid sample, replace check 4's redundant assertions with a family-wide walk that fails on the three embed keys at any depth, encode the two arity couplings in the heatmap and delta-list schemas in the stats idiom, and correct every prose sentence that claims an enforcement which does not exist.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with scripts/validate-contract.mjs and run npm run test:contract to inspect the existing checks and valid samples. Add per-primitive invariant validation, replace check 4 with a family-wide walk, update the heatmap and delta-list schemas, and correct prose claims in CONTRACT.md and related schema descriptions. Done means the contract tests fail for the listed violations and pass for valid samples.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, json
Domain
documentation, testing-qa, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.