ai-cfia / ai-cfia/nachet

Detector evaluation fails on partial batches

オープン
#905 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
bug
主要言語
Python
スター
2
フォーク
2
平均マージ
5日 6時間
マージ済み PR(30日)
5

説明

The detector smoke run completed a training step, then failed during evaluation:

ValueError: not enough values to unpack (expected 2, got 1)

A local reproduction showed that, with one image and a batch size of two, metric gathering shortened `orig_size` from `[480, 640]` to `[480]`. Other partial batches lost bounding boxes without raising an error. Full batches passed.

This reproduces with Transformers 5.16.1 and the previously pinned 5.2.0 revision, both using Accelerate 1.12.0.

Setting `eval_use_gather_object=True` did not resolve the reproduction: partial batches lost prediction components instead. Repeated evaluation also reset the gather setting.

Evaluation needs to preserve every image's annotations and keep them aligned with predictions, including in the final partial batch. Add regression coverage and verify with real detector evaluation.

### References

- [Detector trainer](https://github.com/ai-cfia/nachet/blob/main/mlops/training/detector/src/train_detector.py)
- [Earlier gathering report #30357](https://github.com/huggingface/transformers/issues/30357). Closed after object-gather support was added; enabling that option did not resolve our detector reproduction.
- [Related nested-label truncation report #43388](https://github.com/huggingface/transformers/issues/43388). Closed; proposed fixes [#43395](https://github.com/huggingface/transformers/pull/43395) and [#43419](https://github.com/huggingface/transformers/pull/43419) were closed without merging.

Related to #893.

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。