hypercerts-org / hypercerts-org/hypercerts-lexicon

test: schema-validate object literals in doc snippets

Open
#202 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Product: ePDS Tech debt
Dominant language
TypeScript
Stars
12
Forks
8
Avg merge
1h 9m
Merged PRs (30d)
3

Description

Problem

The doc snippet test suite (tests/validate-doc-snippets.test.ts) currently guards against several classes of documentation drift:

  • Type-string ($type) literals that don't exist in any generated lexicon
  • validate() calls with arguments in the wrong order
  • Imports of symbols that don't exist in the package exports
  • Namespace/mapping accesses against symbols that no longer exist

However, it does not schema-validate the actual record literals embedded in the documentation snippets. This means a snippet like:

const receipt = {
  $type: FUNDING_RECEIPT_NSID,
  subject: { uri: "...", cid: "..." },   // ← field doesn't exist on the lexicon
  paidAt: new Date().toISOString(),      // ← field doesn't exist on the lexicon
  // ... other fields
};

…passes all current tests even though the record is invalid against org.hypercerts.funding.receipt. This exact failure mode was caught by a human reviewer on #196 (the subject/paidAt funding receipt bug fixed in 44c0343), demonstrating the gap concretely.

Proposal

Extend tests/validate-doc-snippets.test.ts with a new check that:

  1. Extracts object literals from TypeScript code blocks in README.md and .agents/skills/building-with-hypercerts-lexicons/SKILL.md.
  2. Identifies each literal's lexicon via its $type marker (either a string literal or a resolved NSID constant).
  3. Runs each literal through the runtime validate() function from generated/lexicons.
  4. Reports failures with source context (file, line, snippet excerpt) to make fixing easy.
Implementation sketch

A possible helper signature:

interface ExampleLiteral {
  obj: unknown;
  nsid: string;          // resolved from $type
  sourceFile: string;
  sourceContext: string; // ~10 lines of surrounding snippet
}

function collectExampleObjectLiterals(
  source: string,
  sourceFile: string,
  nsidConstants: Record<string, string>, // NSID name → value mapping
): ExampleLiteral[];

The new test suite then iterates these and calls validate(obj, nsid, 'main', false) from generated/lexicons.js, expecting result.success === true.

Extraction challenges

Naively regex-extracting object literals is fragile — nested braces, backtick strings, and comments can all confuse a regex. Two plausible approaches:

(a) Regex + bracket counting: Find const <name> = { or <name>: { openers, then walk forward counting {/} while skipping string/comment contents. Simple but brittle around edge cases.

(b) Lightweight TS parsing: Use a small parser (the typescript package is already a transitive dependency; @babel/parser is another option) to AST-walk each block. More robust but adds parse overhead and a direct dependency on a parser.

Given the docs snippets are mostly straightforward const x = { ... } literals, approach (a) is probably sufficient and avoids a new dependency. Approach (b) is only worth it if (a) turns out to misparse real snippets.

NSID resolution

$type in doc snippets is usually one of:

  • A string literal: $type: "org.hypercerts.funding.receipt"
  • An NSID constant: $type: FUNDING_RECEIPT_NSID
  • A dotted namespace access: $type: HYPERCERTS_NSIDS.FUNDING_RECEIPT

The test already parses the import statements (to verify symbols exist), so extending that parse to build a name→value map of imported NSID constants is straightforward.

Why this matters

  • The funding-receipt snippet was wrong for a long time without being noticed.
  • Every added/fixed docs snippet is essentially trusted code for downstream users who copy-paste it.
  • The rest of the test file already establishes the pattern of "lint the docs against the generated code" — this is the logical next step.

Scope

  • tests/validate-doc-snippets.test.ts only.
  • No changes to the lexicons or generated code.
  • No changes to production code.

Out of scope

  • Validating snippets in other markdown files (ERD.md, SCHEMAS.md, changelogs). Can be added later once the core extractor is in place.
  • Validating non-object-literal record construction (e.g. const r = makeReceipt(...)). Only direct object literals are in scope.

References

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tests/validate-doc-snippets.test.ts and review its existing import, symbol, and $type checks, then inspect TypeScript blocks in README.md and .agents/skills/building-with-hypercerts-lexicons/SKILL.md. Add extraction and NSID resolution for direct object literals, validate each with generated/lexicons.js, and include file, line, and snippet context when validation fails.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
documentation, testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.