microsoft / microsoft/mimalloc
How mimalloc behave when allocating a really large memory block?
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 13.4k
- Forks
- 1.2k
- Avg merge
- 4d 45m
- Merged PRs (30d)
- 13
Description
I am now working with some database projects. I found that in the case of Clickhouse, instead of trusting the allocator, they do mmap/mremap on their own if they want to allocate a really large block of memory (bigger than 64MB, and can expand to several GiBs).
While I do not quite understand why this is needed (AFAIK, 64MiB is still a valid size class for snmalloc and mimalloc), I do observe that doing so will actually reduce the performance of large hashtables (tons of syscalls during rehashing). So I begin to wonder why `mmap/mremap` are favored by them? and how do modern allocators like mimalloc handle large allocation in the user space?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file, test, or entry point is named. Start by tracing mimalloc's handling of allocations above 64MB and compare it with the ClickHouse mmap/mremap approach; done means documenting why the strategies differ and how large allocations behave.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- operating-systems, performance
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100