microsoft / microsoft/winml-cli

[Task] keypoint-detection model support

Open
#284 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

model / task scale P2 triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

Overview

Keypoint detection (a.k.a. human pose estimation) models predict the locations of anatomical landmarks — joints, facial landmarks, hand keypoints — on people in images or video frames. This enables downstream reasoning about body posture, gesture, and movement.

Agent Scenarios

  • Fitness / sports coaching agent: analyze exercise form in real time, flag incorrect posture or track rep counts from a webcam feed
  • Accessibility agent: interpret sign language or body gestures as input to a conversational agent
  • AR / avatar agent: drive avatar animations by mapping detected pose to a virtual skeleton
  • Safety monitoring agent: detect falls or abnormal postures in elderly care or industrial environments

ModelKit Integration

Models must pass the full wmk pipeline on all EPs:

wmk config → wmk build (ONNX export) → wmk perf → wmk eval

Acceptance Criteria

  • usyd-community/vitpose-plus-base
  • usyd-community/vitpose-base-simple
  • usyd-community/vitpose-plus-small
  • usyd-community/vitpose-plus-large
  • usyd-community/vitpose-plus-huge
  • stanfordmimi/synthpose-vitpose-huge-hf

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the wmk config, build, perf, and eval pipeline described in the issue, then inspect how existing models are integrated. Add support for the six listed keypoint-detection models and verify that each passes the full pipeline, including ONNX export.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.