facebookresearch / facebookresearch/detectron2

[Feature Request] Iterable-style dataset support for inference_on_dataset

Open
#4,189 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature
I'd like to request adding support for `inference_on_dataset` to handle iterable-style datasets.

## Motivation & Examples

This would allow running evaluation on non-COCO style datasets like TorchData iterables.

I think these type of datasets are gaining in popularity, so support would be useful to Detectron2 users.

This might require a new cfg default to set the number of eval samples to iterate through, maybe for example `TEST.NUM_SAMPLES`. Alternatively, maybe there could be a first pass over the iterable to find its length and then use that for the `total` value.

Contributor guide

Open the contributing guide

Research direction

Start by locating the inference_on_dataset entry point and examining how it obtains dataset length and reports evaluation progress. Compare the expected behavior for TorchData-style iterables, then clarify whether evaluation uses a configured sample limit or a discovered length; done means iterable datasets can be evaluated without requiring a COCO-style length.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.