Nullus157 / Nullus157/async-compression

Consider rusty_zstd as an optional pure-Rust zstd backend

Open
#480 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
666
Forks
118
Avg merge
17h 18m
Merged PRs (30d)
9

Description

### The value of integrating

async-compression provides adaptors between compression crates and Rust's modern asynchronous IO types.

You already depend on the C-backed `zstd` crate; `Cargo.toml` pins `zstd = "^0.13.1"`.

If you added rusty_zstd as an optional backend, that path drops the C toolchain: no `zstd-sys`, no FFI, empty crate tree, and `#![deny(unsafe_code)]` except one audited SIMD island — which matters when the bytes are untrusted. It builds as-is on wasm32 (you already ship wasm32-wasip1-threads) and on `no_std + alloc`. Streaming stays covered; compressed size is within ~2-4% of C (rusty_zstd README). Your async IO adaptors and other algorithm features stay untouched.

Not asking to replace the existing `zstd` feature.

### How to integrate

Add `rusty_zstd` from crates.io behind an optional feature. Wire streaming compress/decompress into the same adaptor types the `zstd` feature uses today. See the rusty_zstd README for the codec API and current version.

Your public async IO surface, tokio/futures features, and tests for the other algorithms stay the same. A feature-flagged trial is enough to judge cost.

### Remade With Rust & MATA

We are rebuilding the stack in safe Rust — codecs, parsers, compression, identity, storage — so projects can drop C dependencies without giving up the format. The rest of the org is at https://github.com/Remade-With-Rust if anything else fits the build. Close this issue if rusty_zstd is a poor fit.

- x.com/farmer_timmm

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with Cargo.toml and the existing zstd feature and adaptor types used for streaming compression and decompression. Read the rusty_zstd README for its codec API and current version, then compare the existing zstd tests and feature wiring. Done means an optional rusty_zstd backend works through the same streaming adaptors without changing the existing zstd path.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, wasm
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.