microsoft / microsoft/onnxruntime
Overflow detection for quantized/half/mixed precision models
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 179
Description
Is your feature request related to a problem? Please describe.
It is common that many issues and unexpected model outputs from quantized, fp16 or mixed precision models are caused by overflowing. It would be much faster and easier to pinpoint the outstanding operator, if there is some degree of overflow detection support on computation/operator level.
Describe the solution you'd like
In case there is a concern for performance impact, the overflow detection can be set enabled only at debug build.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by locating computation and operator execution paths for quantized, fp16, and mixed-precision models, then investigate how a debug-only overflow check could identify the outstanding operator. Done should include operator-level overflow detection with an understood performance impact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100