pytorch / pytorch/pytorch

[Accelerator] Hardware native support versus Software simulated support for Device Capabilties

Open
#169,277 10 comments 0 reactions 0 assignees View on GitHub
module: accelerator triaged
Dominant language
Python
Stars
103k
Forks
29.5k
PR merge metrics
PR metrics pending

Description

### 🚀 The feature, motivation and pitch

This question is purposed by @guangyey in this PR #165631.

# Background

This PR #165631 introduces a new API `torch.accelerator.get_device_capability` for users to get some device capabilities (currently focus on data types) to solve some issues like #165038 and #143112. The supported data types in PyTorch are:
- float32, float64, float16, bfloat16, float8_e5m2, float8_e4m3fn, float8_e5m2fnuz, float8_e4m3fnuz, float8_e8m0fnu, float4_e2m1fn_x2
- complex32, complex64, complex128
- uint1 to uint7, int1 to int7, uint8, int8, int16, int32, int64, uint16, uint32, uint64
- bool
- quint8, qint8, qint32, quint4x2, quint2x4
- bits8, bits16, bits4x2, bits2x4, bits1x8

The API returns a dictionary (which is now only contains the supported data types listed above) of the device capabilities, and it comes up with a question, **should we distinguish the data type is software emulation or hardware native support?**

For fixing issues like #165038 and #143112, we don't need to distinguish them. However, in some performance sensitive cases, this question should be under consideration.

# Purposed changes

## Option 1
**Seperate the returned data types into two categories: hardware support and software support**.
This manner can match all the end users' needs, but it would be more complicated since we have to clarify which data type is hardware natively support on which hardware.

## Option 2
**Regardless of hardware support and software support**
This is an easy way to accomplished.

### Additional context

If anyone has any good idea, please discuss under this post. Thank you!

cc @albanD @guangyey @EikanWang @fffrog @malfet @ngimel @jeffdaily @jgong5

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.