camunda / camunda/api-test-generator
chore: promote RDF spike's index-parity check to a regression invariant
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 3
- Avg merge
- 13h 41m
- Merged PRs (30d)
- 23
Description
Context
The RDF spike (#60) graduated with a phase-3 checkpoint that re-materialises three of graphLoader's reverse indexes from SPARQL queries against an oxigraph triple store, and asserts byte-level parity against the live loader output. The script lives on spike/rdf-issue-60 at:
docs/spikes/rdf/parity/index-parity.tsdocs/spikes/rdf/adapters/build-store.tsdocs/spikes/rdf/ontology/{core,camunda}.ttldocs/spikes/rdf/shapes/invariants.shapes.ttl
It is currently a one-shot CLI script that prints PARITY: PASS/FAIL. There is no CI gate enforcing parity — if the loader's output drifts from what the ontology can express, nothing complains.
#66 (Spike #60 follow-ups) lists this as the last remaining backlog item.
Goal
Promote the parity check to a permanent regression invariant under tests/regression/, so that any future change to graphLoader, domain-semantics.json, the ontology, or the adapter that breaks the loader↔store correspondence fails CI.
Scope
This is a sizeable graduation, not a thin wrapper:
-
Bring spike assets into main:
- Cherry-pick
spike/rdf-issue-60sodocs/spikes/rdf/{adapters,ontology,parity,shapes,...}lands onmain. - Decision needed: keep under
docs/spikes/(signals "still a spike, just type-checked & gated") or relocate to a non-docs/location liketests/regression/rdf-parity/{adapters,ontology}/(signals "permanent test asset"). The latter is more honest about the lifecycle.
- Cherry-pick
-
Add runtime deps:
oxigraph ^0.5.x,n3 ^2.xto thepath-analyserworkspace (or to a newrdf-parityworkspace if we want to keep the SDK boundary clean). -
Wrap as vitest: a new
tests/regression/rdf-parity.test.tsthat:- Calls
loadGraph(baseDir)andbuildStore(baseDir). - Computes the three index diffs (
bySemanticProducer,domainProducers,providerMap). - Strict ontology-rejects policy (changed from spike default): a divergence whose key is
'undefined','', or'null'is a failure, not a tolerated artifact. The spike originally surfaced these as warnings on the assumption SHACL would catch them at load time — but SHACL is not enforced anywhere onmain, and the loader silently writingdomainProducers['undefined']is itself a defect class we want to guard against. Test mustexpect(diffs).toEqual([])after the strict filter.
- Calls
-
Class-scoped guard rationale (per AGENTS.md): the test must reject the class of defect "loader and ontology disagree about which operation produces which type/state", not just the three specific reverse indexes the spike happened to query. The three current SPARQL queries cover the BFS-visible surface; if a new BFS-visible reverse index is added to
OperationGraphlater, this test will not yet cover it. A short comment + checklist in the test header should call that out so reviewers add a fourth SPARQL query when extending the loader. -
Pre-push integration: confirm vitest can drive the script under the existing
npm testflow without a network or filesystem dependency beyond the bundled spec. The spike script readspath-analyser/operation-dependency-graph.json+path-analyser/domain-semantics.json, both produced bynpm run testsuite:generate. The CI step ordering already runstestsuite:generatebeforenpm test, so the test can assume those files exist.
Open design questions
- Workspace placement of the adapter + ontology + test (see point 1).
oxigraphruntime cost: it's a native module (Rust → wasm/native). Confirm CI resolves the right binary on Linux x64 without a pre-build step. If not, either pin to the wasm build or add a CI install step.- Performance budget: spike ran the parity check in <2s locally on the bundled spec. Document a soft ceiling (e.g. 10s on CI) and fail loud if it regresses by 5×.
- Ontology evolution: when a new domain-semantics field lands (e.g. a future
capabilities[T].witnessesshape), the adapter and one or more SHACL shapes will need updates. Document this in the test's header comment so it's discoverable.
Acceptance criteria
-
docs/spikes/rdf/(or relocated equivalent) is onmainand type-checks under the relevant workspacetsconfig.json. -
oxigraph+n3are declared deps in the chosen workspace andnpm ciresolves them on Linux CI. -
tests/regression/rdf-parity.test.tsruns innpm testand asserts byte-level parity forbySemanticProducer,domainProducers,providerMap. - The strict ontology-rejects-fail policy is in place (no
undefined/''/nullkey tolerance). - The test header documents the class-scoped guard rationale and the "add a SPARQL query when adding a reverse index" extension contract.
- CI passes; pre-push checklist (lint + 3× tsc + testsuite:generate + generate:request-validation + npm test) is green locally.
Out of scope
- Replacing the loader with a SPARQL-backed implementation (the spike's longer-term hypothesis). Parity-as-gate is a stepping stone: it locks in the loader↔ontology correspondence so a future refactor can swap the implementation under it without regressing.
- Promoting the SHACL shapes (
invariants.shapes.ttl) to a load-time gate — separate follow-up. - Promoting the other spike queries (
minimal-scenario-chain,value-binding-drift) — separate follow-ups.
Related
- #60 (original spike)
- #66 (spike follow-ups, this is the last remaining item)
- spike branch:
spike/rdf-issue-60 - spike commit:
34524b5 chore(spike-rdf): phase 2+3 — adapters + index-parity checkpoint PASS
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing docs/spikes/rdf/parity/index-parity.ts, docs/spikes/rdf/adapters/build-store.ts, the ontology and shapes, plus the relevant workspace package and CI configuration. Then trace how the generated graph files are produced before designing tests/regression/rdf-parity.test.ts around the existing vitest flow. Done means the assets and dependencies resolve in CI, strict parity checks cover the three named indexes, and npm test passes without network access.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ci-cd, databases, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100