microsoft / microsoft/winml-cli
[Task] keypoint-detection model support
Open
Nobody has claimed this yet.
model / task scale
P2
triaged
- Dominant language
- Python
- Stars
- 40
- Forks
- 11
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 50
Description
Overview
Keypoint detection (a.k.a. human pose estimation) models predict the locations of anatomical landmarks — joints, facial landmarks, hand keypoints — on people in images or video frames. This enables downstream reasoning about body posture, gesture, and movement.
Agent Scenarios
- Fitness / sports coaching agent: analyze exercise form in real time, flag incorrect posture or track rep counts from a webcam feed
- Accessibility agent: interpret sign language or body gestures as input to a conversational agent
- AR / avatar agent: drive avatar animations by mapping detected pose to a virtual skeleton
- Safety monitoring agent: detect falls or abnormal postures in elderly care or industrial environments
ModelKit Integration
Models must pass the full wmk pipeline on all EPs:
wmk config → wmk build (ONNX export) → wmk perf → wmk eval
Acceptance Criteria
- usyd-community/vitpose-plus-base
- usyd-community/vitpose-base-simple
- usyd-community/vitpose-plus-small
- usyd-community/vitpose-plus-large
- usyd-community/vitpose-plus-huge
- stanfordmimi/synthpose-vitpose-huge-hf
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the wmk config, build, perf, and eval pipeline described in the issue, then inspect how existing models are integrated. Add support for the six listed keypoint-detection models and verify that each passes the full pipeline, including ONNX export.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- machine-learning, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100