[Accelerator] Hardware native support versus Software simulated support for Device Capabilties
- Dominant language
- Python
- Stars
- 103k
- Forks
- 29.5k
- PR merge metrics
- PR metrics pending
Description
### 🚀 The feature, motivation and pitch
This question is purposed by @guangyey in this PR #165631.
# Background
This PR #165631 introduces a new API `torch.accelerator.get_device_capability` for users to get some device capabilities (currently focus on data types) to solve some issues like #165038 and #143112. The supported data types in PyTorch are:
- float32, float64, float16, bfloat16, float8_e5m2, float8_e4m3fn, float8_e5m2fnuz, float8_e4m3fnuz, float8_e8m0fnu, float4_e2m1fn_x2
- complex32, complex64, complex128
- uint1 to uint7, int1 to int7, uint8, int8, int16, int32, int64, uint16, uint32, uint64
- bool
- quint8, qint8, qint32, quint4x2, quint2x4
- bits8, bits16, bits4x2, bits2x4, bits1x8
The API returns a dictionary (which is now only contains the supported data types listed above) of the device capabilities, and it comes up with a question, **should we distinguish the data type is software emulation or hardware native support?**
For fixing issues like #165038 and #143112, we don't need to distinguish them. However, in some performance sensitive cases, this question should be under consideration.
# Purposed changes
## Option 1
**Seperate the returned data types into two categories: hardware support and software support**.
This manner can match all the end users' needs, but it would be more complicated since we have to clarify which data type is hardware natively support on which hardware.
## Option 2
**Regardless of hardware support and software support**
This is an easy way to accomplished.
### Additional context
If anyone has any good idea, please discuss under this post. Thank you!
cc @albanD @guangyey @EikanWang @fffrog @malfet @ngimel @jeffdaily @jgong5
Contributor guide
Assessment
This issue has not been assessed yet.