Add batch visualization function to `torchvision.utils`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 7.3k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 13
Description
🚀 The feature
Currently vision models commonly return dictionary
model = fasterrcnn_resnet50_fpn_v2(
weights=FasterRCNN_ResNet50_FPN_V2_Weights.COCO_V1,
)
dog_int = io.read_image("dog.jpeg")
batch = f.convert_image_dtype(dog_int)
response = model(batch.unsqueeze(0))
# [{'boxes': tensor([[ 61.9160, 49.2223, 185.7204, 184.4109],
# [143.5477, 143.5992, 175.1468, 184.4712],
# [136.6003, 217.3529, 161.4755, 223.8254]]),
# 'labels': tensor([18, 20, 57]),
# 'scores': tensor([0.9989, 0.1069, 0.0699])}]
This is visualized by torchvision by
dog_image = utils.draw_bounding_boxes(
dog_int, response[0]["boxes"][response[0]["scores"] > 0.3]
)
f.to_pil_image(dog_image)
Motivation, pitch
However, there is no currenly a way to visualize whole batch with a function in utilities, eg.
continuing from previous
with torch.no_grad():
imgs = [img.detach().clone() for _ in range(16)]
model.eval()
response = model(imgs)
# The input is float tensor batch
# output is equal length list of output dictionaries
There is no currenly available method to visualize this batch for user.
I suggest an utils function
def visualize_batch(image_batch: Tensor | list[Tensor], batch_response: list[dict[str, Tensor]], **visualization_arguments):
"""Function visualizes image batch in a suitable grid and returns result as a tensor
Arguments:
image_batch (Tensor): Float tensor batch, internally transform to uint8 for visualization utils
batch_response: List of response dictionaries, works for both mask and rcnn models as well as training batches by inferring correct behavior from dictionary keys.
"""
# ... implementation
where keyword arguments relate to arguments in current utils functions.
Alternatives
User could follow utilities example at example which implements matplotlib function show. This is non-ideal, as users seems to have need for similar functionality without related boilderplate code.
Additional context
I am willing to contribute given green light.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing torchvision.utils visualization functions and the linked visualization example. Compare their arguments with the proposed visualize_batch inputs, then clarify supported response dictionaries, grid behavior, and how completion will be verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100