microsoft / microsoft/mimalloc

v2 performance regression in heavy allocation scenario (a lot of syscalls)

Open
#447 13 comments 7 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
13.4k
Forks
1.2k
Avg merge
4d 45m
Merged PRs (30d)
13

Description

A simple synthetic allocation/deallocation workload code:
```cpp
int main(int, char*[]) {
static constexpr int kNumBuffers = 20;
static constexpr size_t kMinBufferSize = 5 * 1024 * 1024;
static constexpr size_t kMaxBufferSize = 25 * 1024 * 1024;
std::unique_ptr buffers[kNumBuffers];

std::random_device rd;
std::mt19937 gen(rd());
std::uniform_int_distribution<> size_distribution(kMinBufferSize, kMaxBufferSize);
std::uniform_int_distribution<> buf_number_distribution(0, kNumBuffers - 1);

static constexpr int kNumIterations = 1000;
const auto start = std::chrono::steady_clock::now();
for (int i = 0; i < kNumIterations; ++i) {
int buffer_idx = buf_number_distribution(gen);
size_t new_size = size_distribution(gen);
buffers[buffer_idx] = std::make_unique(new_size);
}
const auto end = std::chrono::steady_clock::now();
const auto num_ms = std::chrono::duration_cast(end - start).count();
const auto us_per_allocation = std::chrono::duration_cast(end - start).count() / kNumIterations;
std::cout << kNumIterations << " allocations Done in " << num_ms << "ms." << std::endl;
std::cout << "Avg " << us_per_allocation << " us per allocation" << std::endl;
return 0;
}
```
unfortunately runs significantly slower with mimalloc v2.0.2 on my machine (Ubuntu linux, x86_64 CPU) than with default system allocator.

_example with system default allocator:_
`1000 allocations Done in 3366ms.
Avg 3366 us per allocation`
_example with mimalloc:_
`1000 allocations Done in 9161ms.
Avg 9161 us per allocation`

The issue seems to be caused by lots of mmap/munmap syscalls by mimalloc compared to default allocator:
```
**perf trace -s bazel-bin/third_party/mimalloc/mimalloc_test**
10000 allocations Done in 488ms.
Avg 48 us per allocation

Summary of events:
mimalloc_test (19177), 123062 events, 100.0%
syscall calls total min avg max stddev
(msec) (msec) (msec) (msec) (%)
--------------- -------- --------- --------- --------- --------- ------
munmap 36680 117.243 0.001 0.003 0.052 0.35%
mmap 19973 44.567 0.002 0.002 0.032 0.23%
madvise 4724 7.485 0.001 0.002 0.003 0.21%

**perf trace -s bazel-bin/third_party/mimalloc/stdalloc_test**
10000 allocations Done in 23ms.
Avg 2 us per allocation

Summary of events:
stdalloc_test (19210), 1400 events, 99.3%
syscall calls total min avg max stddev
(msec) (msec) (msec) (msec) (%)
--------------- -------- --------- --------- --------- --------- ------
brk 506 4.638 0.002 0.009 1.376 29.83%
openat 46 0.290 0.004 0.006 0.018 7.19%
mmap 43 0.204 0.003 0.005 0.009 5.86%
munmap 27 0.203 0.005 0.008 0.016 5.48%
```
In such a scenario mimalloc calls mmap _even more than once_ for each malloc call.

The most interesting part is that the issue **does not reproduce with stable** (v1.7.2) mimalloc, hence the regression.

PS. I realize v2.0.2 is a beta version but hope my case will help it to become stable one day. Beside this issue everything else works perfectly for me:)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the synthetic allocation workload and the bazel-bin/third_party/mimalloc/mimalloc_test and stdalloc_test benchmarks. Reproduce the v2.0.2 versus stable v1.7.2 results on Ubuntu, then use perf trace to compare mmap, munmap, and madvise activity; done means the regression and excess syscall behavior are understood and covered by a repeatable benchmark or test.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cpp, linux
Domain
operating-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.