deepseek-ai / deepseek-ai/DeepSeek-VL2
Significant Discrepancies Between Batch Inference and Single Inference in DeepSeekVL2
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 1.8k
- PR merge metrics
- No merged PRs in 30d
Description
**Description:**
I've noticed that when using DeepSeekVL2-tiny, predictions for the same input differ significantly between single inference (processing one input at a time) and batch inference (processing multiple inputs together). The outputs for a single item in a batch are notably different from its output when run alone. I suspect this might relate to how batch normalization or dropout is handled, or a difference in how the data is preprocessed. Any guidance on why this discrepancy occurs, or how to align the results, would be greatly appreciated.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.