facebookresearch / facebookresearch/detectron2

Visualizer: draw_and_connect_keypoints on more than one instance

Open
#4,000 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

I faced an issue when estimating the skeleton of multiple objects when using draw_and_connect_keypoints it expects the input to belong to one instance only. Although that draw_instance_predictions works on multiple instances. However, it makes plotting of bounding boxes which are not needed sometimes when doing Pose Estimation.

The feature would be useful to add a small level of abstraction that helps people to get involved more into Detectron2. When I tried the function for the first time I didn't know that it accepts only one instance while a predictor of a pose estimation model might return more than one instance (multi-person pose estimation). So I wondered if the function can be improved to accept more than one instance as a general case or make another function that works on more than one instance.

For my case, I tried testing this by making a function that accepts a Tensor of size (N, K, 3) where N is the number of instances, K is the number of keypoints instead of (K, 3), and loop over the instances. It worked well for me when I plotted the visualization. And I wonder if this can improve the Visualizer class by making another function specific to multiple instance pose estimation or editing the existing function.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Visualizer.draw_and_connect_keypoints entry point and compare its single-instance input handling with draw_instance_predictions. Determine whether the requested multi-instance pose visualization should extend the existing function or use a separate entry point, then verify that multiple keypoint instances render without unnecessary bounding boxes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.