deepseek-ai / deepseek-ai/3FS

Opt-in rusty_alloc backend for MimallocLib

Open
#428 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
10.2k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

### The value of integrating

3FS is currently pulling in mimalloc for allocation; `src/memory/mimalloc/MimallocLib.cc` includes `mimalloc.h`.

If you added rusty_alloc as an opt-in backend, the same `mi_*` surface would run on a ground-up Rust allocator. A double free aborts on both the owner and cross-thread paths (mimalloc in release does not). USRBIO, CRAQ and the file interface stay as they are. The rusty_alloc README measures lua 0.97×, perl 0.99× and sqlite 1.00× mimalloc's whole-program instructions retired (`LD_PRELOAD`).

Not asking you to change the default; this would sit behind a compile-time flag.

### How to integrate

You already sit mimalloc behind `MemoryAllocatorInterface`. Add `rusty_alloc_ffi` as an off-by-default backend. It is the `mi_*`-compatible C ABI in the rusty_alloc 1.1.6 tree. Today's mimalloc path stays the default.

```toml
rusty_alloc = "1.1.6"
```

Call sites stay `mi_*`. The names do not change. See the rusty_alloc README (Architecture) for the FFI crate. Builders need rustc for that crate; nothing else in the 3FS tree moves. `MemoryAllocatorInterface`, the file API, tests and on-disk format stay as they are.

Turn it on in CI, compare on your own corpus, and drop it with one line if it does not hold up.

### Remade With Rust & MATA

Remade With Rust is rebuilding the stack in safe Rust — codecs, parsers, compression, identity, storage — so projects can drop C dependencies without giving up performance. If anything else in the org fits the 3FS build, it lives at https://github.com/Remade-With-Rust. Close this issue if it is a poor fit.

- x.com/farmer_timmm

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with src/memory/mimalloc/MimallocLib.cc and MemoryAllocatorInterface, then read the rusty_alloc README's Architecture section for its C ABI. Add an off-by-default backend using rusty_alloc 1.1.6, enable it in CI, and verify that existing call sites remain unchanged while allocator comparisons run on the project's corpus.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, rust
Domain
backend, build-system, ci-cd, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.