Heuristic correctness: canonical fixtures and independent oracles
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Go
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Parent: #17
Depends on the evaluation scaffold issue.
Outcome
Validate the mathematical and resolver correctness of matlatl's shipped mechanisms without using matlatl output as its own oracle.
Scope
- Canonical empty, singleton, path, star, cycle, SCC, DAG, disconnected, multi-root, bridge, articulation, and symmetric-tie graph fixtures.
- Resolver fixtures for relative, root-absolute, directory, fragment, alias/wikilink, ambiguous, external, non-note, stale, malformed, and root-escape references.
- Hand-enumerated or independently calculated expectations for reachability, bow-tie classes, hops, HITS, PageRank, navigability scalars, betweenness, articulation points, bridges, trails, backlinks, scent, knowledge gaps, and suggested links.
- Seeded reversible graph and reference mutations with expected deltas.
- Byte-stability and shuffled-input checks.
Constraints
- Binding definitions are the ADRs listed in
docs/research/heuristic-evaluation.md. - Any unexplained correctness mismatch blocks usefulness or agent-outcome claims for that mechanism.
- Floating-point comparison and accumulation order must be explicitly pinned.
Acceptance
- Every shipped mechanism has at least one independent correctness oracle.
- Threshold boundaries, exemptions, caps, tie ordering, and indeterminate cases are covered.
- Invalid universal monotonic assumptions are not encoded as tests.
- Repeated and shuffled-input runs are byte-stable.
- The complete offline correctness suite runs through
task eval:oracle.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the binding definitions in docs/research/heuristic-evaluation.md and review the evaluation scaffold dependency from issue #17. Build the canonical graph and resolver fixtures with independent expectations, then run task eval:oracle. Done means every shipped mechanism has an oracle, boundary and indeterminate cases are covered, and repeated or shuffled-input runs are byte-stable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100