webarkit / webarkit/jsfeatNext

test(fixtures): OpenCV ground-truth harness for algorithms jsfeat does not have

Open
#132 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement tests
Dominant language
TypeScript
Stars
12
Forks
4
Avg merge
16h 24m
Merged PRs (30d)
34

Description

Summary

Add a reproducible harness that produces OpenCV ground-truth fixtures for
algorithms the vendored jsfeat oracle cannot cover, and a loader that asserts
against them from Vitest.

This is the first concrete slice of the "third-party ground truth" strand of #87.

Why

Every correctness guarantee in this repository currently rests on parity against
the vendored original jsfeat (tests/vendor/). That works for the ported
modules and is the right safety net for them — but jsfeat has no GMS, no
TEBLID, no FREAK
. For those, the oracle does not exist and parity testing has
nothing to say.

This matters beyond jsfeatNext. #96 assigns jsfeatNext the role of numeric
reference oracle
for the PureCV Rust port: PureCV validates bit-for-bit
against vectors produced here. An oracle that has never itself been checked
against OpenCV is not an oracle — it is just the first of two implementations
that may both be wrong, with no way to tell.

Proposed design

Generator

scripts/gen-opencv-fixtures.py, using opencv-python, run manually.

It is deliberately not wired into CI or npm ci: no Python dependency enters
the JavaScript toolchain, and CI keeps running against committed fixtures only.

The input image is committed, not regenerated

Today's tests build images from seeded generators (noiseImage(w, h, seed),
cornerScene(...) in tests/properties/helpers.ts). Reproducing those bytes in
Python would mean reimplementing the RNG in a second language, at the very
bottom of the trust chain, where a silent divergence invalidates every fixture
downstream without announcing itself.

Instead: dump the image once from TypeScript, commit it, and have both sides
read the same bytes. The RNG leaves the trust chain entirely.

Keypoints are input, not output

The fixture commits the keypoint set that was fed to OpenCV, and the TypeScript
test feeds the same set to jsfeatNext. A descriptor mismatch is then
unambiguously a descriptor bug — not a detector bug. Without this, the first
failure forces you to work out whether FAST or TEBLID is at fault.

Layout
tests/fixtures/opencv/
  README.md                 # how to regenerate, and with which OpenCV version
  scene-640x480.png         # committed input, referenced by sha256
  integral-basic.json
  <algo>-<params>.json
scripts/gen-opencv-fixtures.py
Fixture schema
{
  "meta": {
    "opencv_version": "4.x.y",
    "algorithm": "teblid",
    "params": { "bits": 256, "scale_factor": 1.0 },
    "image": "scene-640x480.png",
    "sha256": "…"
  },
  "keypoints": [{ "x": 0, "y": 0, "size": 0, "angle": 0 }],
  "descriptors": ["a3f1…"]
}

JSON with descriptors as hex strings, one per keypoint. A compact binary blob
would save maybe half the bytes and cost the diff: when a fixture changes, JSON
shows which keypoints moved, a blob shows only that the file differs. The repo
already stores textual reference values in tests/reference/known-values.test.ts.

The first fixture must be compute_integral_image

Not a new algorithm — cv2.integral. It is pure integer arithmetic that must
match bit-for-bit, so a mismatch indicts the harness (image dump, encoding,
loader) rather than the algorithm under test. If the first fixture were TEBLID, a
mismatch would be ambiguous and nothing downstream could trust the harness.

It also has standalone value: it is an OpenCV cross-check on a function that
already ships, in a release that is already fixing a batch of correctness bugs.

Acceptance criteria

  • scripts/gen-opencv-fixtures.py regenerates every committed fixture from
    the committed image, deterministically.
  • tests/fixtures/opencv/README.md documents the OpenCV version, how to
    install opencv-python, and how to regenerate — enough for a third party.
  • A TypeScript loader validates meta.sha256 against the image it reads, so
    a stale fixture fails loudly rather than asserting against the wrong input.
  • The compute_integral_image vs cv2.integral example is green, bit-exact.
  • package.json gains no dependency; CI runs unchanged.

Out of scope

  • Fixtures for GMS (#134), TEBLID (#135) or FREAK (#80): those belong to their own issues, which
    consume this harness.
  • Running the generator in CI.
  • Replacing the jsfeat parity oracle for modules jsfeat does have.

Related

  • Parent: #87 (third-party ground truth)
  • First consumers: #134 (GMS), #135 (TEBLID) and #80 (FREAK)
  • Divergence policy when bit-exactness proves unreachable: #102
  • Oracle role for the Rust port: #96
  • Plan: docs/features2d-expansion-plan.md

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the proposed layout, scripts/gen-opencv-fixtures.py, and the tests/fixtures/opencv/README.md requirements; inspect the existing TypeScript fixture and Vitest patterns before choosing the loader entry point. Use the committed image and keypoints to produce the integral-basic.json fixture from cv2.integral, validate its SHA-256, and confirm the TypeScript test is bit-exact without adding a package.json dependency.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript
Domain
computer-vision, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.