messense / messense/fasttext-serving
One allocator on musl and aarch64 too
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 62
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
### The value of integrating
fasttext-serving runs jemalloc on a narrow slice of its targets: `src/main.rs` gates the `#[global_allocator]` on `#[cfg(all(unix, not(target_env = "musl"), not(target_arch = "aarch64")))]`. So a musl container, an aarch64 host and any Windows build all fall back to the platform allocator — and the tree also carries `mimalloc = { version = "0.1" }`, meaning there are two allocator crates in the manifest and a majority of modern deployment targets using neither.
For a model-serving process that gap is on the hot path. The service stays up and answers predictions over HTTP (`actix-web`) or gRPC (`tonic = "0.8.0"`), both enabled by `default = ["http", "grpc"]`, on a multi-threaded runtime — so every request allocates input and prediction buffers that are typically freed on a different worker than the one that made them, continuously, for the life of the container.
rusty_alloc is one allocator on all of those targets, with 9,261 downloads on crates.io, in safe Rust with no build script and no `target_env` or `target_arch` conditions to keep in sync. Against jemalloc the rusty_alloc README's callgrind arms measure 0.84x, 0.89x and 0.98x whole-program instructions on lua, perl and sqlite, under `LD_PRELOAD`.
Not asking you to drop jemalloc where it is enabled — this would sit behind an off-by-default feature.
### How to integrate
Following the pattern you already use for `http` and `grpc`:
```toml
[features]
rusty-alloc = ["dep:rusty_alloc-api"] # opt in; `default = ["http", "grpc"]` unchanged
[dependencies]
rusty_alloc-api = { version = "2.1", optional = true }
```
```rust
#[cfg(feature = "rusty-alloc")]
#[global_allocator]
static GLOBAL: rusty_alloc_api::RustyAlloc = rusty_alloc_api::RustyAlloc;
```
Nothing else changes: same REST and gRPC surface, same model loading. The interesting comparison is an aarch64 or musl deployment, since those have no allocator choice at all today.
### Remade With Rust & MATA
We rebuild C dependencies in safe Rust — allocators, codecs, parsers, compression, identity and storage — so a project can take C out of its build without giving up performance. The rest is at https://github.com/Remade-With-Rust. If this does not suit you, please close the issue.
- x.com/farmer_timmm
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the allocator cfg in src/main.rs and the dependency and feature definitions in the manifest. Check how the existing jemalloc and mimalloc setup is selected, then verify the new opt-in feature builds for musl and aarch64 while the default http and grpc features and model-serving surfaces remain unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100