microsoft / microsoft/mimalloc

Cross-Thread Delayed Free Collection Issue in mimalloc

Open
#1,117 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
13.4k
Forks
1.2k
Avg merge
4d 45m
Merged PRs (30d)
13

Description

## Summary

There is a memory leak issue in mimalloc where large blocks freed from different threads may never be properly collected, leading to memory not being returned to the system. This occurs due to a mismatch between how cross-thread delayed frees are queued and how heap collection processes them.

## Root Cause

The issue stems from the interaction between mimalloc's cross-thread freeing mechanism and heap collection logic:

1. **Cross-thread delayed frees are heap-specific**: When a block allocated on Thread A is freed from Thread B, the block gets added to the original allocating heap's `thread_delayed_free` list (not the freeing thread's heap).

2. **Heap collection is single-heap focused**: `mi_heap_collect_ex()` only processes delayed frees for the specific heap being collected, not other heaps that may exist on the same thread.

3. **Multiple heaps per thread**: Each thread can have multiple heaps (linked via `heap->next`), but collection happens independently on each heap.

## Problem Flow

```
1. Thread A allocates large block from Heap H1
2. Thread B frees the block → block queued to H1's thread_delayed_free list
3. Thread A calls mi_heap_collect_ex(H2) or H1 gets abandoned while H2 remains active
4. H1's delayed free list is never processed → memory leak
```

## Code Locations

### Where delayed frees are queued (cross-thread)
**File**: `src/free.c:236-239`
```c
// add to the delayed free list of this heap
mi_block_t* dfree = mi_atomic_load_ptr_relaxed(mi_block_t, &heap->thread_delayed_free);
do {
mi_block_set_nextx(heap,block,dfree, heap->keys);
} while (!mi_atomic_cas_ptr_weak_release(mi_block_t,&heap->thread_delayed_free, &dfree, block));
```

### Where delayed frees are processed (single heap only)
**File**: `src/heap.c:157`
```c
// free all current thread delayed blocks.
_mi_heap_delayed_free_all(heap); // ← Only processes the specific heap being collected
```

### Heap collection entry point
**File**: `src/heap.c:124-180` (`mi_heap_collect_ex`)

## Affected Scenarios

1. **Large block allocations**: Blocks > `MI_LARGE_OBJ_SIZE_MAX` (16MB on 64-bit)
2. **Cross-thread freeing**: Any block freed from a different thread than allocated
3. **Multiple heaps per thread**: Applications using `mi_heap_new()` to create additional heaps
4. **Thread termination**: When threads exit and abandon heaps containing unprocessed delayed frees
5. More obvious in thread pool scenario, where I have 32 cores and 32 threads in the thread pool.

## Reproduction

This issue is most likely to manifest in applications that:
- Use large allocations (>16MB)
- Free memory from different threads than allocated
- Use multiple custom heaps per thread
- Have high thread turnover

## Potential Fix

The fix should ensure that when heap collection happens (especially during `MI_ABANDON`), it processes delayed frees from all relevant heaps on the thread, not just the one being abandoned.

**Suggested approach**: Modify `mi_heap_collect_ex()` to iterate through all heaps in the thread's heap list (`heap->tld->heaps`) and process their delayed free lists when appropriate.

## Environment

- **mimalloc version**: Latest main branch (commit: 09a27098)
- **Platform**: All platforms affected
- **Compilation flags**: Default configuration

## Impact

- Memory not returned to OS despite being "freed"
- Potential memory exhaustion in long-running applications
- RSS memory usage higher than expected
- Most severe with large allocations and multi-threaded usage patterns

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the delayed-free queueing in src/free.c:236-239 and the collection path in src/heap.c:124-180, especially mi_heap_collect_ex and _mi_heap_delayed_free_all. Trace how heap->tld->heaps and MI_ABANDON interact, then verify that delayed frees on relevant heaps are processed and large cross-thread allocations can be returned to the system.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
operating-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.