Accelerating Vector<T> with SVE on ARM64
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
We want to accelerate `Vector` using the Scalable Vector Extension (SVE) on ARM64, which provides instructions that operate on vectors of arbitrary size. Different hardware implementations can support different vector register lengths (in powers of 2 from 128 bits, up to 2048 bits), but the same instruction set is used to operate on them. Software can determine the vector register size by executing the `rdvl` instruction.
The current implementation of `Vector` uses NEON and is fixed to 128 bits across all hardware. However, there are intrinsics available for the class that allow you to execute SVE instructions directly.
The key consequence of this feature is that the size of `Vector` becomes a runtime constant, and therefore cannot be statically evaluated by the compiler. Most of the existing codebase assumes that all types have a known, statically determinable size (typically obtained via `genTypeSize`). We need to introduce a new type classification for `Vector` that may not support this assumption, requiring the compiler to handle it differently.
There are three scenarios we need to consider:
- JIT compilation
- Ahead-of-time (AOT) compilation with a specified target vector length
- Ahead-of-time (AOT) compilation without a known target vector length
Scenarios 1 and 2 are roughly equivalent in complexity. Either the compiler learns the size of `Vector` by executing `rdvl` (Scenario 1: JIT), or the user provides a desired size via a compiler option (Scenario 2: AOT). Both cases can be managed by the EE and communicated to RyuJIT through the JIT–EE contract.
I’ve been exploring a solution for these two scenarios by introducing a new type, `TYP_SIMDSV`, and refactoring areas that process SIMD nodes to query the type size dynamically from the compiler, rather than using `genTypeSize`.
It is not possible to AOT-compile with a specified vector length and then execute the resulting binary on hardware with a different vector length. Assumptions about the size of `Vector` would no longer hold, and memory corruption is likely. This has implications for how users interact with NativeAOT and R2R. The user must ensure that the process’s vector length matches the value used at compile time.
Scenario 3 is more complex, as it requires adding compilation paths that cannot assume any specific vector size. The compiler must generate code sequences that query the vector register size at runtime. Any code generation that depends on vector size, such as stack frame layout or context serialization, must be implemented in a way that is completely independent of the actual value.
Scenarios 1 and 2 can be viewed as optimizations of Scenario 3. When the vector length is known at compile time, the compiler can omit `rdvl` calls and substitute a compile-time constant instead. However, because the JIT currently relies heavily on knowing the size of all internal types, it may be pragmatic to tackle Scenarios 1 and 2 first, and then move toward fully vector-agnostic compilation later.
### Problems to Solve
#### Base Support
- [x] Introduce `TYP_SIMD` to the JIT type system in a manner compatible with NEON
- https://github.com/dotnet/runtime/pull/121114 (initial research)
- https://github.com/dotnet/runtime/pull/121489
- https://github.com/dotnet/runtime/pull/121548
- [x] Implement stack frame allocation for `TYP_SIMD` and `TYP_MASK`, and value classes containing `Vector`
- https://github.com/dotnet/runtime/pull/122638
- https://github.com/dotnet/runtime/pull/125491
- https://github.com/dotnet/runtime/pull/124516
- [x] Support spill temps with types `TYP_SIMD/TYP_MASK`
- https://github.com/dotnet/runtime/pull/127917
- [x] Zero Initialization and frame poisoning of `TYP_SIMD/TYP_MASK` locals
- https://github.com/dotnet/runtime/pull/128148
- [ ] Update ABI and LSRA for parameter passing and non-volatile registers ([AAPCS link](https://github.com/ARM-software/abi-aa/blob/main/aapcs64/aapcs64.rst#613scalable-vector-registers))
- https://github.com/dotnet/runtime/pull/125729
- https://github.com/dotnet/runtime/pull/131746
- https://github.com/dotnet/runtime/pull/131745
- [ ] Port `Vector` implementations to SVE with a NEON fallback when the feature is not available.
- https://github.com/dotnet/runtime/pull/123992
- https://github.com/dotnet/runtime/issues/125057
- https://github.com/dotnet/runtime/pull/125226
- https://github.com/dotnet/runtime/pull/128326
- https://github.com/dotnet/runtime/pull/127520
- https://github.com/dotnet/runtime/pull/133422
- [ ] Update exception handling and unwinding (suspension `CONTEXT` records)
- https://github.com/dotnet/runtime/pull/133152
- [ ] Update testing to scale with vector length and test across all available configurations (NEON only, 128-bit SVE, 256-bit SVE).
- https://github.com/dotnet/runtime/pull/129852
- [ ] Update UnknownSizeFrame with awareness of OSR frames
- https://github.com/dotnet/runtime/pull/130023
- https://github.com/dotnet/runtime/pull/132752
- [ ] Update the debugger tooling with awareness of vector/predicate register state
- [ ] Update support for TYP_MASK to be VL agnostic
- [ ] Remove config variable and default Vector codegen to SVE when available
#### Codegen improvements
- [x] Change vector representation of masks to AllBitSet in active lanes
- https://github.com/dotnet/runtime/pull/128326
- [ ] Addressing modes with VL scalars
- [ ] GC extensions for VL scalars (e.g. `N * VL + Imm`, enables NativeAOT compilation)
- [ ] Introduce mask nodes for 'embedded mask' intrinsics earlier in the pipeline
- [ ] Merge 'optional embedded mask' definition with AVX implementation
- [ ] Load/Store Coalescing
Contributor guide
Assessment
This issue has not been assessed yet.