microsoft / microsoft/mimalloc
Slowdown when using mimalloc to allocate (many) buffers
- Dominant language
- C
- Stars
- 13.4k
- Forks
- 1.2k
- Avg merge
- 4d 45m
- Merged PRs (30d)
- 13
Description
Hello,
I am playing with Apache Arrow, which is using mimalloc (by default), and noticed some weird performance that I could not explain and that goes away when switching to the `system` allocator. I extracted the core of the problem, shown in the following:
```c++
#include
#include
#include
#include
#include
static constexpr std::size_t NumPages = 1000;
static constexpr std::size_t NumElements = 64 * 1024 + 1;
int main(int argc, char *argv[]) {
std::vector> pages;
for (std::size_t i = 0; i < NumPages; i++) {
pages.emplace_back(new float[NumElements]);
}
auto start = std::chrono::steady_clock::now();
for (std::size_t e = 0; e < NumElements; e++) {
for (std::size_t i = 0; i < NumPages; i++) {
pages[i][e] = 0;
}
}
auto end = std::chrono::steady_clock::now();
const std::chrono::duration duration = end - start;
std::cout << "run time: " << duration.count() << "\n";
return 0;
}
```
Compiling with `g++ -O2 pages.cxx -o pages` and running gives:
```
$ taskset -c 2 ./pages
run time: 0.674548
```
However, preloading `mimalloc.so` is 10x slower:
```
$ LD_PRELOAD=install/lib64/libmimalloc.so taskset -c 2 ./pages
run time: 7.09013
```
This is on Alma Linux 8 with a somewhat dated glibc 2.28 and Linux kernel 4.18.0-553.51.1.el8_10.x86_64. However, I can also reproduce a 3x slowdown on Arch Linux with a recent kernel:
```
$ taskset -c 2 ./pages
run time: 0.66908
$ LD_PRELOAD=/usr/lib64/libmimalloc.so taskset -c 2 ./pages
run time: 1.91931
```
The numbers above are from the current `main` (or `v2.2.4` when using the Arch Linux package), but I can equally reproduce with `v1.9.4` and latest `dev3`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by building and running the pages.cxx reproducer with g++ -O2, then compare the system allocator against LD_PRELOAD=install/lib64/libmimalloc.so under the reported CPU affinity. Investigate allocator behavior for the repeated page allocations and traversal, using the Alma Linux and Arch Linux environments as comparison points. Done means identifying the cause of the slowdown and documenting or correcting it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- operating-systems, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100