OOXML: recoverable allocation failure in Package::part is classified as malformed
- Dominant language
- Rust
- Stars
- 21.5k
- Forks
- 1.3k
- Avg merge
- 42m
- Merged PRs (30d)
- 17
Description
## Summary
In `@firecrawl/anydoc@0.2.4`, a recoverable allocation failure while reading an OOXML ZIP part can be reported as `code: "malformed"` with:
```text
malformed document (word/document.xml): corrupt archive entry: out of memory
```
The memory pressure here is deliberately imposed by our subprocess sandbox. This report concerns the error classification when an OOM reaches the error-return path, not a requirement that every native allocation failure be recoverable or that conversion succeed under the limit.
## Environment and observations
- Bun `1.4.1+4661e494f`, Linux x64 GNU, native AnyDoc npm binding `0.2.4`.
- Fresh child per conversion; empty environment; `--smol --no-env-file --config=/dev/null`; input through stdin.
- `RLIMIT_DATA` soft/hard: 536870912 bytes; core dumps disabled; outer GNU `timeout` of 5 seconds. This is an allocation limit, not an RSS cap or an observed kernel OOM kill.
- Synthetic DOCX: 73,241 compressed bytes; `word/document.xml` is 67,109,048 bytes and contains one run with 64 MiB of ASCII `Z` text. The ZIP entries other than that XML part are retained from a valid DOCX.
- Across 120 previously recorded runs of the same input: 117 children terminated with SIGABRT; 3 reached the JS catch with the exact error above (`code: "malformed"`). Stdout was empty in every run. The three caught cases were run indices 2, 93 and 104 (zero-based). This is an observed distribution, not a guaranteed reproduction rate.
The same input is recognized as `docx` before conversion. The unusual error is specifically `out of memory`, not a checksum or XML syntax failure.
## Reproduction setup
Install `@firecrawl/anydoc@0.2.4` in an isolated directory with Bun 1.4.1. Prepare the stress input from any valid `base.docx`:
```python
from zipfile import ZipFile, ZIP_DEFLATED
prefix = b''
suffix = b''
with ZipFile('base.docx') as source, ZipFile('expanded.docx', 'w', ZIP_DEFLATED) as output:
for entry in source.infolist():
data = prefix + b'Z' * (64 * 1024 * 1024) + suffix if entry.filename == 'word/document.xml' else source.read(entry)
output.writestr(entry.filename, data)
```
The original measured fixture's SHA-256 is `9ebc7b1eed5c4ae34863dad3af2da274c57ecd888b34dc9360682536a499153b`; a different base document/ZIP metadata will produce different bytes and may change the allocation outcome.
Use a child that reads stdin, calls `formatFromBytes(bytes)`, then `await toMarkdownBytes(bytes, format, { ocr: 'reject' })`, and logs `error.code` and `error.message` from its catch. Run each child with an absolute Bun path:
```sh
env -i /usr/bin/timeout --foreground --signal=KILL --kill-after=0.1s 5s \
/usr/bin/prlimit --data=536870912:536870912 --core=0:0 -- \
/absolute/path/to/bun --smol --no-env-file --config=/dev/null ./child.ts \
< expanded.docx
```
Allocation timing/reservations affect whether the process aborts or returns an error; one run may only show SIGABRT. The 120-run observation used an application stdin reader before the AnyDoc call, so the simplified harness above is a reproduction setup, not a claim that it reproduces the same frequency.
## Source evidence and expected result
[`Package::part` in v0.2.4](https://github.com/firecrawl/anydoc/blob/v0.2.4/src/package/archive.rs#L63-L68) maps every `read_to_end` error to `ConvertError::Malformed` and prefixes it with `corrupt archive entry`. The same mapping is present in [current main at 261fc257](https://github.com/firecrawl/anydoc/blob/261fc257d17c3eab0f673be31c408fd9fdc2171a/src/package/archive.rs#L63-L68) (source inspection; only 0.2.4 was run).
When a recoverable `std::io::ErrorKind::OutOfMemory` is returned, preserve its allocation/resource-failure meaning instead of classifying the document as corrupt. `ResourceLimit` or another distinct error category would let callers distinguish resource exhaustion from malformed input without matching message text. Ordinary corrupt ZIP errors should remain `Malformed`.
A focused regression could inject an `OutOfMemory` read error and a genuine corrupt-input error, verifying that their categories differ. This does not require an allocator-dependent stress test to return the caught error every time.
Our current workaround recognizes only `code === 'malformed'` together with the exact `: corrupt archive entry: out of memory` suffix; native aborts remain separate subprocess failures.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in src/package/archive.rs at Package::part and inspect how read_to_end errors become ConvertError categories. Add focused coverage for an injected OutOfMemory read error and a genuine corrupt-input error, then run the relevant package tests. Done means resource exhaustion has its own category while ordinary corrupt ZIP errors remain Malformed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100