microsoft / microsoft/mimalloc

High virtual memory usage, reset_decommits doesn't work

Open
#537 17 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
13.4k
Forks
1.2k
Avg merge
4d 45m
Merged PRs (30d)
13

Description

Hi! I'm evaluating using mimalloc-secure 2.0.3 for our game instead of Unreal Engine's default FMallocBinned2.
The runtime performance seems good, I have not measured any significant regression from this switch.
However, it seems to not decommit memory nearly as much if the memory usage of the game goes down.

My configuration:
MI_SECURE = 4

The target for comparison, using FMallocBinned2:
- Game started
![image](https://user-images.githubusercontent.com/6764788/151112949-d421ac0a-1d17-4908-abc3-62f2835d301c.png)
- Loaded large save
![image](https://user-images.githubusercontent.com/6764788/151112978-571f4a04-2bd5-483a-85af-e71a6632ef5a.png)
- Cleared large save
![image](https://user-images.githubusercontent.com/6764788/151112992-06602b0f-c2b6-4bbd-8321-24988b9fafa3.png)
- Loaded large save again
![image](https://user-images.githubusercontent.com/6764788/151113007-7483fb1c-4181-4c38-951b-366607e3b4c7.png)

The initial results using mimalloc:
- Game started
![image](https://user-images.githubusercontent.com/6764788/151113096-d907ac4d-c5ee-4b16-9391-fe265a360610.png)
- Loaded large save
![image](https://user-images.githubusercontent.com/6764788/151113108-564c2bb8-15cf-42be-88d9-f6977e2024fb.png)
- Cleared large save
![image](https://user-images.githubusercontent.com/6764788/151113120-0900d09b-0822-403c-9795-58738bcc1399.png)
- Loaded large save again
![image](https://user-images.githubusercontent.com/6764788/151113130-fa589ea3-81e2-41e6-8dd0-4ef65673addb.png)

That's... not great in comparison.

I enabled the options page_reset, abandoned_page_reset, gives significant improvements for the working set, but no improvement on unnecessary memory being decommitted again.
- Game started
![image](https://user-images.githubusercontent.com/6764788/151113297-116d35fc-0985-494b-a340-48b65f7a49cc.png)
- Loaded large save
![image](https://user-images.githubusercontent.com/6764788/151113306-9ffac52f-33c2-4144-b7dc-60062555211c.png)
- Cleared large save
![image](https://user-images.githubusercontent.com/6764788/151113319-ca5b411e-4a65-4a5f-8f37-fa3016fb735d.png)
- Loaded large save again
![image](https://user-images.githubusercontent.com/6764788/151113329-238386e9-09a6-420b-b960-dcb1a0fbe77f.png)

The option segment_reset seems to not be hooked up to anything.
The option reset_decommits causes the game to crash instantly during static initialization, if page_reset is also enabled (otherwise it does nothing):
![image](https://user-images.githubusercontent.com/6764788/151113611-e92f077d-6f6c-44ec-abb7-537ca67b3647.png)
![image](https://user-images.githubusercontent.com/6764788/151113692-bbc841b8-82ff-4996-afd9-bf8936cdd327.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the handling for page_reset, abandoned_page_reset, segment_reset, and reset_decommits, then reproduce the static-initialization crash with MI_SECURE=4 and page_reset enabled. Compare the decommit behavior after memory usage falls. Done means reset_decommits no longer crashes or silently does nothing, and the reported decommit behavior is covered by validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.