JdeRobot / JdeRobot/VisualCircuit
New CV block proposal: MediaPipe Hand & Gesture detector
- Dominant language
- JavaScript
- Stars
- 20
- Forks
- 34
- Avg merge
- 4h 56m
- Merged PRs (30d)
- 10
Description
Hi @jmplaza — following the Block Library Extension (#69), I'd like to contribute a new
Computer Vision block and wanted to confirm the scope before packaging it.
### Block: Hand & Gesture Detector (MediaPipe Hand Landmarker)
VisualCircuit has FaceDetector (Haar) and ObjectDetector (YOLO) but no hand / landmark /
gesture block. This fills that gap and pairs naturally with the Controller blocks
(MotorDriver, Teleoperator) for gesture-driven robotics.
**What it does:** runs MediaPipe's Hand Landmarker on the input image to detect up to two
hands (21 landmarks each) and estimate a simple gesture (extended-finger count).
**Inputs:** BGR image (`Img`)
**Outputs (via `OutputType` parameter):**
- `image` — frame with hand landmarks, connections and a gesture label drawn on it
- `landmarks` — flat array of 21 (x, y) normalised landmarks + finger count, so downstream
blocks can react to gestures
**Parameter:** `OutputType` ('image' / 'landmarks')
**Implementation notes:**
- Uses MediaPipe's current **Tasks API** (`mediapipe.tasks.vision.HandLandmarker`) — the
legacy `mp.solutions.hands` API was removed in MediaPipe 0.10.30+, so the Tasks API is the
durable choice.
- Ships a bundled `hand_landmarker.task` model under `utils/models/`, mirroring how
FaceDetector/ObjectDetector bundle their Haar/YOLO models. One new dependency: `mediapipe`.
- Mirrors the existing block API (`inputs.read_image` / `outputs.share_image` /
`outputs.share_array` / `synchronise`).
I've prototyped and tested it (detects both hands and draws the 21-point skeleton on a real
frame). Once you're happy with the scope, I'll build it in the editor, export the `.vc3`, and
open a PR on VisualCircuit-resources with a short demo video per the custom-block guide.
Does this fit the library? Happy to adjust the I/O or naming.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the existing FaceDetector and ObjectDetector blocks, the custom-block guide, and the model location under utils/models/. Verify the MediaPipe Tasks API and the inputs.read_image, outputs.share_image, outputs.share_array, and synchronise entry points. Done means the new block supports both OutputType values, bundles hand_landmarker.task, and is exported as a .vc3 with the requested demo.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, robotics
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100