linebender / linebender/fearless_simd
`dispatch!` is expensive
- Dominant language
- Rust
- Stars
- 457
- Forks
- 30
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 25
Description
In [PhastFT](https://github.com/QuState/PHastFT/) for smaller sizes I'm calling `dispatch!` three times when running an FFT operation on 512 bytes of data (64-long batch of f64) and it is degrading performance by 25% (-20% throughput) measured as of commit https://github.com/QuState/PhastFT/tree/e5fcd61f3d540fcef9f8d60173dbfbe777c02e40
Meanwhile RustFFT with its handwritten dispatch does not suffer any penalty at all, and in fact is slightly slower under `-C target-cpu=native` than it is under its regular dynamic dispatch.
This overhead needs to be removed for code based on `fearless_simd` to be competitive with handwritten dynamic dispatch.
`perf diff` and profiling with `samply` both point to these `dispatch!` calls as a major source of slowdown: https://github.com/QuState/PhastFT/blob/c7ea3d7aef474e53233834354364fa50bbb0ba6e/src/algorithms/dit.rs#L259-L260
Profile with `-C target-cpu=x86-64-v3`: https://share.firefox.dev/3LMqjuI
Profile with dynamic dispatch: https://share.firefox.dev/3NTwJZw
I'm not sure what the cause is. I wouldn't expect a handful of perfectly predictable branches to tank performance. Perhaps `dispatch!` results in subotimal codegen, or perhaps I'm just pushing the boundaries of dynamic dispatch and need a facility to get a function pointer and store it in a struct for reuse instead of just reusing a cached `Level`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the dispatch! calls in src/algorithms/dit.rs around lines 259-260, then reproduce the 512-byte FFT benchmark at commit e5fcd61f3d540fcef9f8d60173dbfbe777c02e40. Use perf diff and samply with the provided profiles to identify the overhead and compare against the dynamic-dispatch case. Done means removing the dispatch! performance penalty for small FFT operations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100