huggingface / huggingface/evaluate

add_batch taking very long time (SegFormer semantic segmentation).

Open
#513 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

When I call **"add_batch"** with 512X512 prediction image (and 512X512) label
With batch size of 40 (so the input are 40X512X512).
It takes **50 seconds per batch** on my machine (windows 10 with RTX 3090 and i9-12900K intel CPU. Torch: 1.12.1+cu113).
(It is stuck in "encode_batch" which call the recursive "encode_nested_example")
It is much longer than training itslef

As a patch:
**I see that If I switch to "Evaluate.compute" it still takes long time but "Evaluate._compute" is much faster.**

**Can I change "add_batch" to something else to enjoy the faster execution the same way replacing "compute" with "_compute ?**

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.