KillingSpark / KillingSpark/zstd-rs

compress/compress_to_vec fails with 'memory allocation of 131072 bytes failed' in no_std microcontroller environment

Open
#94 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
449
Forks
66
PR merge metrics
No merged PRs in 30d

Description

The messages range from 5 to 500 bytes but the same failed allocation results:

```
panic_handler: panicked at library/alloc/src/alloc.rs:438:13:
memory allocation of 131072 bytes failed
```

I've tried making my heap larger than 131072 but that still fails.

```
use alloc::vec:::Vec;

...
use ruzstd::encoding::{compress_to_vec, CompressionLevel};
let msg_bytes = self.encode()?; // get alloc vec Vec

let compressed = compress_to_vec(&*msg_bytes, CompressionLevel::Fastest);

// let mut compressed = Vec::new();
// compress(&*msg_bytes, &mut compressed, CompressionLevel::Fastest);
```

`CompressionLevel::Uncompressed` also fails the same.

Elsewhere decompresing works fine:

```
let decoder = ruzstd::decoding::StreamingDecoder::new(buf).unwrap();

// buf is &[u8]
let mut decode_buf = Vec::new();
let rv = decoder.read_to_end(&mut decode_buf);
...
```

The same code tests fine in the non-embedded standard library environment. I'm new to embedded rust so perhaps I have something configured wrong with my heap.

I notice 131072 comes from here: https://github.com/KillingSpark/zstd-rs/blob/master/src/encoding/blocks/compressed.rs#L268

I can do `let output: &mut Vec = &mut Vec::with_capacity(1024 * 130);` as in `compress()` once in my code and it works fine with my 200k heap.

I can

I'm using `0.8.1`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at src/encoding/blocks/compressed.rs around line 268 and compare the allocation paths used by compress_to_vec and compress. Reproduce the failure in the no_std microcontroller setup with 5–500 byte messages and CompressionLevel::Uncompressed, then verify compression works within the available heap without the 131072-byte allocation.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
embedded-iot
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.