facebookresearch / facebookresearch/perception_models

Tips for Using PE Spatial on TACO Dataset (Object Detection)

Open
#75 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.4k
Forks
162
PR merge metrics
No merged PRs in 30d

Description

Hi, and thank you for releasing this great work.

I'm working on adapting **PE Spatial** for object detection on the [TACO dataset](https://github.com/pedropro/TACO) (Trash Annotations in Context), which has 60+ object classes of litter in real-world scenes.

I’ve reviewed the paper and code, but I’d appreciate some pointers on how best to proceed with the following:

1. Is there a preferred pipeline or example config for finetuning PEspatial on a custom dataset like TACO?
2. Should I extract intermediate features from a specific layer, or is the aligned output of PEspatial already suitable for detection with standard decoders (e.g., ViTDet + Mask R-CNN)? Is it possible to use YOLO?
3. Any recommendations on input resolution, training schedule, or data augmentations that transfer well from COCO to a smaller dataset like TACO?
4. Would it be better to freeze the backbone or allow full finetuning given the domain shift?

If there's a public example or finetuning script available (even for COCO), that would help me a lot in getting started.

Thanks again for the release, looking forward to exploring PE further!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.