apache / apache/arrow-rs

`arrow-ipc`: a footer block's declared body length is zero-filled before it is read

Open
#10,921 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

### Describe the bug

`arrow_ipc::reader::read_block` (reader.rs:922) sizes its buffer from the
footer's `Block` before reading:

let body_len = block.bodyLength().to_usize().unwrap();
let metadata_len = block.metaDataLength().to_usize().unwrap();
let total_len = body_len.checked_add(metadata_len).unwrap();
let mut buf = MutableBuffer::from_len_zeroed(total_len);
reader.read_exact(&mut buf)?;

A 1,018-byte file whose footer holds a record-batch block with
`bodyLength = 0x280000000240` (44 TB) reaches `from_len_zeroed`, which
ASan reports as allocation-size-too-big and which aborts the process on
a normal build. `read_exact` would have returned `UnexpectedEof` had the
allocation been bounded — the check exists one line too late.
`FileReader::try_new` verifies the footer's flatbuffer but not that its
blocks lie within the file.

### To Reproduce

```
cargo new ipc-repro && cd ipc-repro
cargo add arrow-ipc@59.2.0
# copy the attached block-body.arrow beside Cargo.toml

// src/main.rs
fn main() {
let file = std::fs::File::open("block-body.arrow").unwrap();
let reader = arrow_ipc::reader::FileReader::try_new(file, None).unwrap();
for batch in reader {
println!("{:?}", batch.map(|b| b.num_rows()));
}
}
```

cargo run --release

### Expected behavior

`Err(ArrowError::IpcError(..))` — the block is beyond the file.

### Additional context

`FileReader` knows the file's length (it seeks to the footer). Checking
each block's `offset + metaDataLength + bodyLength <= file_len` when the
footer is parsed — or reading with a length-capped `take` into a growing
buffer in `read_block` — turns this into an error. The `MessageReader`
path (reader.rs:1845, `from_len_zeroed(message.bodyLength())`) has the
same shape for the stream format.

Linux x86_64, Rust stable 1.98.0. Found by fuzzing a reader built on the
crate.

Contributor guide

Open the contributing guide

Research direction

Read reader.rs at read_block around line 922 and the MessageReader path around line 1845, then run the provided cargo reproduction with the attached block-body.arrow file. Check how FileReader::try_new obtains the file length and how malformed block lengths are handled. Done means the crafted file returns ArrowError::IpcError without attempting an oversized allocation, including the stream path.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.