[RFC] More flexible floating-point type specification
- Dominant language
- C++
- Stars
- 1.2k
- Forks
- 169
- PR merge metrics
- No merged PRs in 30d
Description
### Motivation
There are a large number of floating-point types in DLPack, [especially 4-bit and 8-bit variants](https://github.com/dmlc/dlpack/blob/84d107bf416c6bab9ae68ad285876600d230490d/include/dlpack/dlpack.h#L163-L181). The structure to define new FP types for DLPack exchange currently requires modifications to DLPack itself, and this RFC aims to relax that limitation.
MLIR moved to a more flexible system in the beginning of last year ([discussion](https://discourse.llvm.org/t/rethink-on-approach-to-low-precision-fp-types/82361), [PR](https://github.com/llvm/llvm-project/pull/118891)), and I feel we should follow in their footsteps.
@seberg [commented in a different issue](https://github.com/dmlc/dlpack/issues/184#issuecomment-4335618705) that a way forward without ABI breakage would be a requirement to do this short-term, and I agree.
### Proposal
This RFC proposes a few things, that when taken together, allow one to specify many¹ floating point types.
1. The introduction of two new enum values in the [`kDLDataTypeCode` enum](https://github.com/dmlc/dlpack/blob/84d107bf416c6bab9ae68ad285876600d230490d/include/dlpack/dlpack.h#L138-L182): kDLFlexibleFloat and kDLFlexibleComplex.
2. That the [`DLDataType` struct](https://github.com/dmlc/dlpack/blob/84d107bf416c6bab9ae68ad285876600d230490d/include/dlpack/dlpack.h#L184-L215) be turned into a (C-style) union. The existing layout would become the first variant of the struct, and the second variant is specified below.
```c
typedef struct {
/*!
* \brief Type code of base types.
* We keep it uint8_t instead of DLDataTypeCode for minimal memory
* footprint, but the value should be one of DLDataTypeCode enum values.
* */
uint8_t code;
/*!
* \brief Number of bits minus one, common choices are 2, 4, 8, 16, 32, 64.
* For complex, specify the number of bits for the components rather than the whole.
*/
uint8_t bitsm1;
/*!
* \brief Number of mantissa bits.
* Implies number of exponent bits are bits - 1 - mantissa in the common case.
* mantissa = 0 specifies an exponent-only format (one example is float8_e8m0fnu)
*/
uint8_t mantissa;
/*!
* \brief The flags associated with this FP type.
* Beginning with the LSB:
* Bit 0: Finite bit, often specified by f
* Bit 1: NaN bit, often specified by n
* Bit 2: (Signed) zero bit, often specified by z
* Bit 3: Unsigned zero bit, often specified by uz
* Bits 5-7: Reserved.
*/
uint8_t flags;
/*! \brief Specifies the number of lanes in the type, used for vector types.
* The value should be raised to a power of two, i.e., a value of 5 means 32 lanes.
*/
uint8_t p2lanes;
} DLDataTypeFloatingSpec;
```
This variant of the struct is only activated by the two enum values above. The existing enum and equivalents here would be considered aliases to each other, but implementations are not required to support the latter if they already support the former.
### Open Questions
The `float8_e4m3b11fnuz` type is not covered by this proposal in its current form. I'm currently debating how best to address that. Options are:
1. Leave it as out of scope/requiring its own enum value.
2. Split `p2lanes` into two 4-bit fields, one for `p2lanes` and the other for `bias`.
3. Use the reserved bits for `bias` instead.
4. Use a combination to increase the bits available to `bias`.
¹ I would consider non IEEE-style types such as POSIT and TAKUM to be explicitly excluded from the scope of this proposal, perhaps to be addressed by future flags.
cc @seberg and @hpkfft as they showed interest.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the kDLDataTypeCode enum and DLDataType definition in include/dlpack/dlpack.h, then read the linked MLIR discussion and issue #184 comment for compatibility context. The open questions, especially float8_e4m3b11fnuz and ABI constraints, need resolution before an implementation can be considered done.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, cpp
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100