Optional pure-Rust allocator for the worker executor
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 1.5k
- Forks
- 212
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 119
Description
The value of integrating
golem-worker-executor/src/server.rs sets mimalloc unconditionally — static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc;, with mimalloc = "0.1.52". That is a considered choice for a process whose whole job is holding many live workers, and it is also the one dependency in the executor that has to be built by a C toolchain everywhere you ship.
rusty_alloc is the same algorithm without that: mimalloc v2.4.5 rebuilt in safe Rust, covering about 150 of mimalloc's roughly 157 mi_* entry points, with the C original kept in-tree only as a differential oracle. It has 8,851 downloads on crates.io and no build script.
The executor is the right place to measure it. It keeps WebAssembly components resident so they never lose state, with rayon = "1.11.0", crossbeam and dashmap in the tree and a multi-threaded tokio runtime underneath — so allocations routinely outlive the thread that made them and are freed elsewhere. On exactly that path the rusty_alloc README measures 0.83x mimalloc's instructions (xthread — every free performed by a non-owning thread). It also aborts on a double free rather than accepting it, which for a host process running untrusted guest components is a difference in failure mode, not just in speed.
Not asking you to change the default — this would sit behind an off-by-default feature.
How to integrate
[features]
rusty-alloc = ["dep:rusty_alloc-api"] # opt in; mimalloc stays the default
[dependencies]
rusty_alloc-api = { version = "2.1", optional = true }
#[cfg(feature = "rusty-alloc")]
#[global_allocator]
static GLOBAL: rusty_alloc_api::RustyAlloc = rusty_alloc_api::RustyAlloc;
#[cfg(not(feature = "rusty-alloc"))]
#[global_allocator]
static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc;
Nothing else changes: same durability guarantees, same gRPC surface, same component model.
Remade With Rust & MATA
We are working through the common C dependencies of the Rust ecosystem and rebuilding them in safe Rust — allocators, compression, codecs, parsers, identity, storage. Everything is at https://github.com/Remade-With-Rust, in case something there suits you better than this does. If it is a poor fit, please close the issue.
- x.com/farmer_timmm
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with golem-worker-executor/src/server.rs and the executor manifest, then inspect the existing mimalloc dependency and global allocator setup. Verify the optional feature and both allocator configurations with the executor's available build or test commands. Done means the default still uses mimalloc, the opt-in configuration compiles with rusty_alloc-api, and no other executor behavior or interfaces change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, distributed-systems, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 66/100