microsoft / microsoft/mimalloc

mi_thread_stats_print_out shows incorrect values for Release builds

Open
#1,083 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
13.4k
Forks
1.2k
Avg merge
4d 45m
Merged PRs (30d)
13

Description

Hi, I'm using mimalloc 2.2.3 built through vcpkg on a C++ program running on Windows. I want to use mimalloc's per-thread stats to monitor my threads' memory consumption on a long-running program which is built in Release mode. However, when using `mi_thread_stats_print_out` I see the values are incorrect (i.e. much lower than they should be).

As an example, I have the following code:

```
#include
#include
#include

#include
#include

static void thread_fn_leak()
{
for (auto i = 0; i < 5; ++i)
{
std::this_thread::sleep_for(std::chrono::seconds(5));

new char[104857600];

mi_thread_stats_print_out(nullptr, nullptr);

puts("");
}
}

int main()
{
mi_version();

std::thread t(thread_fn_leak);

t.join();

return 0;
}
```

When built on Debug mode, the stats look correct:

```
heap stats: peak total current block total#
bin 1: 8 B 8 B 8 B 8 B 1 not all freed
bin 13: 160 B 160 B 160 B 160 B 1 not all freed
bin 28: 2.0 KiB 2.0 KiB 2.0 KiB 2.0 KiB 1 not all freed
bin 33: 5.0 KiB 5.0 KiB 5.0 KiB 5.0 KiB 1 not all freed

heap stats: peak total current block total#
binned: 7.1 Ki 7.1 Ki 7.1 Ki not all freed
huge: 520.0 Mi 520.0 Mi 520.0 Mi not all freed
total: 520.0 MiB 520.0 MiB 520.0 MiB
malloc req: 6.0 KiB

reserved: 0 0 0 ok
committed: 0 0 0 ok
reset: 0
purged: 0
touched: 520.4 MiB 520.4 MiB 520.4 MiB
segments: 6 6 6 not all freed
-abandoned: 0 0 0 ok
-cached: 0 0 0 ok
pages: 9 9 9 not all freed
-abandoned: 0 0 0 ok
-extended: 9
-retire: 0
arenas: 0
-rollback: 0
mmaps: 0
commits: 0
resets: 0
purges: 0
guarded: 0
threads: 0 0 0 ok
searches: 0.0 avg
numa nodes: 1
elapsed: 25.076 s
process: user: 0.000 s, system: 0.000 s, faults: 1638, rss: 6.0 MiB, commit: 574.5 MiB
```

However, when built on Release mode I see the following:

```
heap stats: peak total current block total#
reserved: 0 0 0 ok
committed: 0 0 0 ok
reset: 0
purged: 0
touched: 385.5 KiB 385.5 KiB 385.5 KiB
segments: 6 6 6 not all freed
-abandoned: 0 0 0 ok
-cached: 0 0 0 ok
pages: 0 0 0 ok
-abandoned: 0 0 0 ok
-extended: 0
-retire: 0
arenas: 0
-rollback: 0
mmaps: 0
commits: 0
resets: 0
purges: 0
guarded: 0
threads: 0 0 0 ok
searches: 0.0 avg
numa nodes: 1
elapsed: 25.054 s
process: user: 0.000 s, system: 0.000 s, faults: 1477, rss: 5.4 MiB, commit: 554.5 MiB
```

Here `touched` shows 385 KB when in reality the thread is allocating around 500 MB. This is correctly reported by the (presumably OS-provided) `process` stats, but I'd like to have to have thread-specific stats here too since my real program uses thread pools, etc.

Am I doing something wrong, or is this expected to work correctly only for Debug builds?

Thanks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the supplied Windows C++ reproducer and compare mi_thread_stats_print_out output between Debug and Release builds. Trace the implementation of mi_thread_stats_print_out and the per-thread statistics it reports, then verify that allocations made by the worker thread produce accurate thread-specific values in Release mode.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cpp
Domain
operating-systems, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.