deepseek-ai / deepseek-ai/DeepSeek-VL2

Significant Discrepancies Between Batch Inference and Single Inference in DeepSeekVL2

Open
#129 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
5.4k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

**Description:**
I've noticed that when using DeepSeekVL2-tiny, predictions for the same input differ significantly between single inference (processing one input at a time) and batch inference (processing multiple inputs together). The outputs for a single item in a batch are notably different from its output when run alone. I suspect this might relate to how batch normalization or dropout is handled, or a difference in how the data is preprocessed. Any guidance on why this discrepancy occurs, or how to align the results, would be greatly appreciated.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.