[discossuion] Contemplating task schedulers on NUMA systems
- Dominant language
- Rust
- Stars
- 4.1k
- Forks
- 339
- PR merge metrics
- No merged PRs in 30d
Description
First of all this isn't a solution or a request for a specific feature but rather I'd like to kick of an discussion in the hope that it has some fruitful results.
## Disclaimer
I did dig into this topic a bit after experiencing very odd performance on a Ryzen 3000 systems and am somewhat fascinated by how the chaplet architecture affects performance. The observation I made had to do with how cores communicate but go hand in hand on how NUMA or multi CPU systems **should** behave - but I do not have a multi core system at hand to verify this.
## What
With the task scheduler might spawn and schedule tasks on different threads it will be affected by NUMA architectures. It would be interesting and likely beneficial to performance, to explore how the schedule can take architecture like this into account.
The biggest impact I could see is moving tasks from one to another core to another or communicating between tasks on different cores.
## Why
Non single die systems become more prominent, with Ryzen multi die cpu's that to a degree are NUMA systems, have started to appear in community hardware. However those considerations will probably apply to multi cpu systems as well.
Aside of memory access, which might not affect tasks as much, the impact of not having a shared cache can be huge. I've seen it being in the excess of double (or half depending on how you look) performance difference when having shared or not having shared.
For numbers - pinning threads to cores that share a cache has moved throughput in a benchmark from 135MB/s to over 400MB/s.
Taking cache invalidation into consideration when communication between non cache sharing cores share crossbeam channels get up to 2x faster (https://github.com/crossbeam-rs/crossbeam/pull/462) in some scenarios.
Just a few thoughts :)
Contributor guide
Assessment
This issue has not been assessed yet.