webarkit / webarkit/jsfeatNext
examples: end-to-end AR demo anchoring a 3D object via pose_estimator
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 12
- Forks
- 4
- Avg merge
- 16h 24m
- Merged PRs (30d)
- 34
Description
Summary
Now that pose_estimator exists (#83), an example should show the full natural-feature AR pipeline end to end: detect → describe → match (#133 bfmatcher) → homography (motion_estimator) → camera pose (R, t) → a 3D object anchored on the target.
Today the ORB samples (sample_orb.html, sample_orb_pinball.html) stop at the homography: find_transform fits H and render_pattern_shape warps the target outline into a flat 2D quad. pose_estimator provides the missing H → [R | t] step, but nothing demonstrates it.
This is the visual counterpart to #83, deliberately split out (same way #176 followed #133).
Why this is its own issue, not part of #83
Rendering a 3D pose needs renderer glue that jsfeatNext deliberately does not ship. pose_estimator.estimate stops at (R, t) in the OpenCV camera frame. Putting a 3D object on screen needs:
modelViewGL(pose)— a column-major 4×4 modelview for WebGL/three.js, with the CV→GL axis flip (diag(1, -1, -1)).projectionGL(K, near, far)— a GL projection from the intrinsics.
#83 moved these up to the high-level AR layer's renderer adapters (#97) on purpose, to keep jsfeatNext renderer-agnostic. The reference implementations are kept in #83's prototype <details> block for whoever writes this.
So the example itself carries that glue (it's an example, not library code) — which means this should land after #96/#97 settle where the adapter boundary is, so the example's glue matches the eventual adapter rather than being written now and rewritten later.
Prerequisite (same as #176)
The examples load ../dist/jsfeatNext.mjs, and dist/ is only rebuilt at release time (per MAINTAINERS.md; #143 tracks dropping it from version control). So pose_estimator reaches the example after a release that includes #83, or alongside a deliberate local dist/ rebuild for verification.
Sketch
Two shapes to weigh at implementation:
- A new
examples/sample_pose.html— cleanest: a dedicated demo that tracks a planar target and anchors a simple 3D object (a cube/axes) on it via three.js or raw WebGL. Doesn't disturb the existing ORB samples. - Extend
sample_orb_pinball.html— reuse its existing tracking, add the pose + render step. Less new code, but conflates two demos.
Either way the pose math is estimator.estimate(H, pose); the new work is the renderer glue and a minimal 3D scene, plus pose_estimator.intrinsics(w, h) for the uncalibrated bootstrap.
Acceptance criteria
- An example anchors a 3D object on a tracked planar target using
jsfeat.pose_estimator. - The CV→GL renderer glue (
modelViewGL/projectionGL) lives in the example, not insrc/— jsfeatNext stays renderer-agnostic. - Uses
pose_estimator.intrinsics(w, h)for the bootstrapK, or documents where a real calibration would plug in. - Verified live in a browser with a webcam (like #177).
Related
- Provides the pose primitive: #83
- Sibling example follow-up (bfmatcher): #176 / #177
- Where the renderer-adapter boundary is decided: #96, #97
- dist/ prerequisite: #143
- Plan:
docs/roadmap-1.0.0.md
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
First review #96/#97 and the prototype details in #83 to confirm the renderer-adapter boundary, then inspect sample_orb.html and sample_orb_pinball.html for the existing tracking flow. Verify that the released or locally rebuilt dist/jsfeatNext.mjs exposes pose_estimator. Done means a browser webcam demo anchors a 3D object to a planar target, keeps modelViewGL/projectionGL in the example, and uses or documents intrinsics(w, h).
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ar-vr-xr, computer-vision, web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100