block / block/buzz

Larger .xlsx/.docx uploads are stored as `.zip` — the 4 KiB sniff window truncates OOXML detection

Open
#2,964 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

**Describe the bug**

`file_mime_to_ext()` (`crates/buzz-media/src/validation.rs:99`) maps
`…spreadsheetml.sheet → xlsx`, `…wordprocessingml.document → docx`, `…presentationml… → pptx`.
Those arms are reachable only for *small* files. A realistic spreadsheet is classified as
`application/zip` and stored and served as `.zip`.

The cause is an interaction between two layers, not a wrong mapping:

1. `process_buffered_upload` sniffs a bounded prefix — `MIN_SNIFF_BYTES = 4096`
(`crates/buzz-media/src/upload.rs:351`).
2. `infer` 0.19's `msooxml` matcher (which backs `is_xlsx`/`is_docx`, registered *ahead* of
`application/zip`) is positional: after matching `docProps` / `[Content_Types].xml` /
`_rels/.rels` at `0x1E`, it walks to the **third local file header** to read the
subdirectory name that distinguishes `xl/` from `word/` from `ppt/`.

In real OOXML files the theme and styles parts are large, so that third header sits well past
4 KiB. The matcher's bounded search finds nothing inside the sniff buffer, gives up, and the
file falls through to `application/zip`.

**To Reproduce**

Generate two spreadsheets that differ only in row count (`pip install openpyxl`):

```python
from openpyxl import Workbook

def make(path, rows):
wb = Workbook(); ws = wb.active
for _ in range(rows):
ws.append(["lorem ipsum"] * 8)
wb.save(path)

make("small.xlsx", 5) # 3rd local header at ~2.6 KB — inside the sniff window
make("large.xlsx", 500) # 3rd local header at ~12 KB — outside it
```

```
buzz upload file --file small.xlsx → "url": ".../.xlsx" ✅
buzz upload file --file large.xlsx → "url": ".../.zip" ❌
```

Both files have identical entry order (`docProps/app.xml`, `docProps/core.xml`,
`xl/theme/theme1.xml`, …) and are valid spreadsheets that open in Excel. Only the size of the
parts differs, which moves the header the matcher depends on.

Reproduced on a real 23 KB report as well (3rd header at offset 20,800). Not a caching
artefact: re-zipping the same content to get a fresh sha behaves the same.

**Expected behavior**

Both store as `.xlsx`. The current behaviour is the wrong way round — *toy* files are detected
correctly and *real* ones are not, so it looks fine in testing and fails in use.

**Why it matters**

The stored extension is what downstream consumers see. Anything forwarding Buzz attachments
onward delivers `report.zip`. It still opens, so nothing is
broken — but to a non-technical recipient a `.zip` reads as "something went wrong". It also
feeds `serve_inline` / content-disposition decisions, which key off the same MIME.

**Additional context**

The server cannot fall back to a hint: `PUT /upload` carries raw bytes, and the Blossom auth
event (kind 24242) carries only `t`, `x`, `expiration`, `server` — no filename, no declared
type. So this is not fixable from the client.

Two directions:

- **Raise the sniff window for the zip family only.** Once `infer` says `application/zip`,
re-run OOXML detection over a larger prefix (the third header is bounded by the size of the
first two entries, so ~64 KiB would cover ordinary documents). Cheap, and keeps everything
else at 4 KiB.
- **Inspect the zip central directory** for an `xl/`, `word/` or `ppt/` prefix. Deterministic
and independent of both entry ordering and part sizes, unlike `infer`'s positional walk. The
central directory lives at the *end* of the file, so this needs the tail rather than a bigger
head — worth noting for the streaming path.

Either way the sniff must stay authoritative for allow/deny; this only disambiguates *within*
a family that `BLOCKED_FILE_MIME_TYPES` has already let through.

**Environment**

- Relay: self-hosted, `main` @ `ab3af828`
- `infer` 0.19 (as pinned in `crates/buzz-media/Cargo.toml`)
- macOS 15 (arm64)

Contributor guide

Open the contributing guide

Research direction

Start in crates/buzz-media/src/upload.rs at process_buffered_upload and MIN_SNIFF_BYTES, then trace file_mime_to_ext in crates/buzz-media/src/validation.rs and the pinned infer 0.19 behavior. Reproduce with the small and large openpyxl spreadsheets, then choose and validate an OOXML disambiguation approach. Done means realistic .xlsx, .docx, and .pptx files retain their extensions while ordinary ZIP files and allow/deny decisions remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.