[NEP10 NEP38 NEP54] SIMD and Iterator Performance

Overview

NumPy's approach to SIMD optimization and iterator performance through universal intrinsics and runtime dispatch.

---

NEP 10: Optimizing Iterator/UFunc Performance

Status: Final | Full Text

Memory Layout Options

NPY_ANYORDER = -1    // F if all inputs F, else C
NPY_CORDER = 0       // C-contiguous (row-major)
NPY_FORTRANORDER = 1 // Fortran-contiguous
NPY_KEEPORDER = 2    // Match input layout (NEW)

Cache-Coherency (order='K')

Preserve input memory layout instead of forcing C-contiguous:

a.T + b.T  # 8.3x faster with order='K'

Dimension Coalescing

Merge adjacent contiguous dimensions for single-loop iteration:

If strides[i+1] * shape[i+1] == strides[i]:
    Merge dimensions i and i+1

Buffering

Copy chunks to cache-friendly buffer for poor memory layouts:

• Up to 19x speedup for non-contiguous arrays

Casting Modes

NPY_NO_CASTING        // Identical types only
NPY_EQUIV_CASTING     // + byte-swapped
NPY_SAFE_CASTING      // Safe casts only
NPY_SAME_KIND_CASTING // + same-kind casts
NPY_UNSAFE_CASTING    // Any casts

---

NEP 38: SIMD Universal Intrinsics

Status: Final | Full Text

Three-Stage Mechanism

1. Infrastructure: Abstract intrinsics in code
2. Compile-time: Compiler converts to concrete intrinsics
3. Runtime: CPU detection selects optimal loop

Universal Intrinsic Concept

npyv_load_u32  → vld1q_u32 (NEON) / _mm256_loadu_si256 (AVX2)
npyv_add_f32   → vaddq_f32 (NEON) / _mm256_add_ps (AVX2)

Supported Instruction Sets

• x86_64: SSE3 (baseline), SSE4, AVX, AVX2, AVX-512
• ARM: NEON
• PowerPC: VSX

Build Options

• --cpu-baseline: Minimum required features
• --cpu-dispatch: Additional dispatch variants

---

NEP 54: Google Highway Adoption

Status: Accepted | Full Text

Why Highway

• C++ (cleaner than C intrinsics)
• Sizeless SIMD support (ARM SVE, RISC-V RVV)
• Extensive documentation
• Used by Chromium, JPEG XL

Code Improvement

// Old C
npyv_@sfx@ a5 = npyv_load_@sfx@(src1 + npyv_nlanes_@sfx@ * 4);

// New C++ with Highway
auto a5 = Load(src1 + nlanes * 4);

NumPy 2.0 Usage

Sorting functions accelerated via Highway/Intel x86-simd-sort.

---

Suggested Implementation for NumSharp

.NET SIMD Options

API	.NET Version	Portability	Control
Vector<T>	Core 2.0+	High	Low
Vector128/256/512<T>	Core 3.0+	Medium	High

Priority Operations

Operation	Expected Speedup
Element-wise (+, -, *, /)	4-8x
Comparisons	4-8x
Reductions (sum, mean)	2-4x
Dot/matmul	4-16x

Implementation Pattern

// Runtime dispatch
ISimdBackend backend = Avx2.IsSupported ? new Avx2Backend()
                     : Avx.IsSupported ? new AvxBackend()
                     : Sse2.IsSupported ? new Sse2Backend()
                     : new ScalarBackend();

Iterator Enhancements

• Implement dimension coalescing
• Add order='K' output allocation
• Consider buffered iteration for non-contiguous arrays

---

Related Issues

• #544, #545

Documentation

See docs/neps/NEP10.md, docs/neps/NEP38.md, docs/neps/NEP54.md