`arrow-ipc`: a footer block's declared body length is zero-filled before it is read
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
### Describe the bug
`arrow_ipc::reader::read_block` (reader.rs:922) sizes its buffer from the
footer's `Block` before reading:
let body_len = block.bodyLength().to_usize().unwrap();
let metadata_len = block.metaDataLength().to_usize().unwrap();
let total_len = body_len.checked_add(metadata_len).unwrap();
let mut buf = MutableBuffer::from_len_zeroed(total_len);
reader.read_exact(&mut buf)?;
A 1,018-byte file whose footer holds a record-batch block with
`bodyLength = 0x280000000240` (44 TB) reaches `from_len_zeroed`, which
ASan reports as allocation-size-too-big and which aborts the process on
a normal build. `read_exact` would have returned `UnexpectedEof` had the
allocation been bounded — the check exists one line too late.
`FileReader::try_new` verifies the footer's flatbuffer but not that its
blocks lie within the file.
### To Reproduce
```
cargo new ipc-repro && cd ipc-repro
cargo add arrow-ipc@59.2.0
# copy the attached block-body.arrow beside Cargo.toml
// src/main.rs
fn main() {
let file = std::fs::File::open("block-body.arrow").unwrap();
let reader = arrow_ipc::reader::FileReader::try_new(file, None).unwrap();
for batch in reader {
println!("{:?}", batch.map(|b| b.num_rows()));
}
}
```
cargo run --release
### Expected behavior
`Err(ArrowError::IpcError(..))` — the block is beyond the file.
### Additional context
`FileReader` knows the file's length (it seeks to the footer). Checking
each block's `offset + metaDataLength + bodyLength <= file_len` when the
footer is parsed — or reading with a length-capped `take` into a growing
buffer in `read_block` — turns this into an error. The `MessageReader`
path (reader.rs:1845, `from_len_zeroed(message.bodyLength())`) has the
same shape for the stream format.
Linux x86_64, Rust stable 1.98.0. Found by fuzzing a reader built on the
crate.
Contributor guide
Research direction
Read reader.rs at read_block around line 922 and the MessageReader path around line 1845, then run the provided cargo reproduction with the attached block-body.arrow file. Check how FileReader::try_new obtains the file length and how malformed block lengths are handled. Done means the crafted file returns ArrowError::IpcError without attempting an oversized allocation, including the stream path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100