facebookresearch / facebookresearch/detectron2
[Feature Request] Iterable-style dataset support for inference_on_dataset
- Dominant language
- Python
- Stars
- 34.7k
- Forks
- 7.9k
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
I'd like to request adding support for `inference_on_dataset` to handle iterable-style datasets.
## Motivation & Examples
This would allow running evaluation on non-COCO style datasets like TorchData iterables.
I think these type of datasets are gaining in popularity, so support would be useful to Detectron2 users.
This might require a new cfg default to set the number of eval samples to iterate through, maybe for example `TEST.NUM_SAMPLES`. Alternatively, maybe there could be a first pass over the iterable to find its length and then use that for the `total` value.
Contributor guide
Research direction
Start by locating the inference_on_dataset entry point and examining how it obtains dataset length and reports evaluation progress. Compare the expected behavior for TorchData-style iterables, then clarify whether evaluation uses a configured sample limit or a discovered length; done means iterable datasets can be evaluated without requiring a COCO-style length.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100