microsoft / microsoft/onnxruntime
[Feature request] Support bfloat16/float8 inputs in `session.run()`
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
`session.run()` currently does not support bfloat16 inputs because numpy does not support bfloat16. Would it be possible to support the input via `np.uint16`? `session.run()` can accept an uint16 value which is the bit representation of the bfloat16 value. ORT should be able to interpret the value correctly because it knows the expected input type of the graph. Same can be done for the float8* types.
Contributor guide
Research direction
Start at the session.run() input handling and trace how input types are checked against the graph's expected types. Verify the behavior for uint16 bit representations of bfloat16 and the float8 types, then add coverage showing that these inputs are interpreted correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, numpy, python
- Domain
- api, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100