microsoft / microsoft/mimalloc

_mi_stats_main.committed.current does goes out of sync with real RSS usage

Open
#670 5 comments 1 reaction 0 assignees View on GitHub
stale
Dominant language
C
Stars
13.4k
Forks
1.2k
Avg merge
4d 45m
Merged PRs (30d)
13

Description

When tracking `_mi_stats_main.committed.current ` I would expect that it won't go bigger than `VmRSS` in `/proc/../status`.

In practice, I see that for a long running process, I get:
```
comitted_memory:98553561088
used_memory_rss:89664389120
```
which does not make sense to me. We regularly run `mi_stats_merge()` from all the threads to combine the stats into `_mi_stats_main.`

Aggregating data from
`mi_heap_visit_blocks(heap, false, ..)` over all threads provides accurate information about committed memory. It is similar to what I would expect to see (close to used_memory_rss).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with _mi_stats_main and mi_stats_merge, then compare their committed-memory accounting with aggregation from mi_heap_visit_blocks across threads. Reproduce the long-running discrepancy against VmRSS and trace which accounting path causes the values to diverge. Done means the reported committed memory is explained and corrected or the intended relationship to VmRSS is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.