firecrawl / firecrawl/pdf-inspector
Rotated column headers are scrambled across cells when the page is not mostly rotated
- Dominant language
- Rust
- Stars
- 19.1k
- Forks
- 1.3k
- Avg merge
- 9h 21m
- Merged PRs (30d)
- 51
Description
## Summary
Glyphs of a 90°-rotated text run are never recombined. Each glyph keeps its own device position, and since a rotated run advances along y at a fixed x, line grouping drops one glyph into every row band. A rotated column header comes out reversed, and two rotated lines sharing a column come out interleaved character by character.
`correct_rotated_page` handles pages that are *entirely* rotated. A handful of rotated headers on an otherwise horizontal page — the ordinary case for a table with narrow columns — sits far below its two-thirds vote threshold and is left as-is.
On a corpus of 30 technical PDFs this produces **58 scrambled cells across 3 documents**. Another extractor run over the same corpus produces 0.
## Reproduction
Deterministic, no sample file needed. `make_repro_rotated.py`:
```python
from reportlab.lib import colors
from reportlab.lib.pagesizes import A4
from reportlab.pdfgen import canvas
from reportlab.platypus import Table, TableStyle
# Column 1 carries two rotated lines, the case that interleaves rather than
# merely reversing. Offsets are in units of COL_W plus a within-column shift.
HEADERS = [(1, 0.0, "Inlet pressure"), (1, 10.5, "static"), (2, 0.0, "Flow rate"),
(3, 0.0, "Temperature")]
ROWS = [
["Mass flow", "3,10E+00", "3,35E-01", "4,23E-01"],
["Heat output", "2,41E+02", "2,74E+01", "6,69E+00"],
["Drop", "1,91E+03", "3,96E+02", "8,67E+01"],
["Power input", "4,52E+01", "9,03E+00", "1,22E+00"],
["Efficiency", "8,80E-01", "7,40E-01", "6,10E-01"],
["Runtime", "2,00E+03", "1,50E+03", "9,00E+02"],
]
FONT, SIZE, ADVANCE = "Helvetica", 9, 6.0
LEFT, TOP, COL_W, ROW_H, HEADER_H = 60.0, 700.0, 60.0, 14.0, 90.0
table = Table([[""] * 4] + ROWS, colWidths=[COL_W] * 4, rowHeights=[HEADER_H] + [ROW_H] * len(ROWS))
table.setStyle(
TableStyle(
[
("GRID", (0, 0), (-1, -1), 0.5, colors.black),
("FONTNAME", (0, 0), (-1, -1), FONT),
("FONTSIZE", (0, 0), (-1, -1), SIZE),
]
)
)
c = canvas.Canvas("repro_rotated_headers.pdf", pagesize=A4)
table.wrapOn(c, A4[0], A4[1])
table.drawOn(c, LEFT, TOP - HEADER_H - ROW_H * len(ROWS))
# Headers sit inside the tall first row, reading bottom-to-top.
c.setFont(FONT, SIZE)
for column, shift, header in HEADERS:
c.saveState()
c.translate(LEFT + COL_W * column + 12.0 + shift, TOP - HEADER_H + 4.0)
c.rotate(90)
for position, glyph in enumerate(header):
c.drawString(position * ADVANCE, 0, glyph)
c.restoreState()
c.save()
```
```
python make_repro_rotated.py
cargo run --release --bin pdf2md -- repro_rotated_headers.pdf /dev/stdout
```
**Expected**
```
||Inlet pressure static|Flow rate|Temperature|
|---|---|---|---|
|Mass flow|3,10E+00|3,35E-01|4,23E-01|
```
**Actual**
```
||e r u s s e r p c t i e t l a n t I s|e t a r w o l F|e r u t a r e p m e T|
|---|---|---|---|
|Mass flow|3,10E+00|3,35E-01|4,23E-01|
```
The first cell is `Inlet pressure` and `static` woven together, one character at a time. Body rows are unaffected — the damage is confined to the header, which is what binds each column of numbers to its meaning.
## Cause
Every glyph of a rotated run is emitted as its own `TextItem`. With a 90° matrix the advance falls entirely on device y, so the run is a column of items at one x:
```
164.25 300.21 w= 0.00 fs= 9.00 "I"
164.25 311.75 w= 0.00 fs= 9.00 "n"
164.25 321.86 w= 0.00 fs= 9.00 "l"
174.75 300.21 w= 0.00 fs= 9.00 "s" <- second rotated line, same column
174.75 309.71 w= 0.00 fs= 9.00 "u"
```
Nothing puts them back together. `correct_rotated_page` (`src/extractor/content_stream.rs:1315`) is the only rotation handling, and it bails unless most of the page is rotated:
```rust
let total_votes = votes.horizontal + votes.rotated;
if total_votes == 0 || votes.rotated * 3 < total_votes * 2 {
// Less than ~67% of text operators are rotated → not a rotated page
return (items, rects, lines, false);
}
```
A table with a dozen rotated headers over a page of horizontal body text never reaches that. Reading order is also lost independently of grouping: these files draw rotated glyphs visually top-to-bottom, which for 90° CCW is reverse reading order, so even a single-line header emerges backwards.
## Suggested fix
A pass over the extracted items, before line grouping and before `merge_text_items` — which otherwise steals loose glyphs into neighbouring horizontal text — that groups zero-width single-glyph items sharing a baseline x, orders them by ascending y, and emits one item per run positioned at the run's start with the glyph body left of the baseline.
Zero width is the signal that a run is rotated, and it comes for free: the advance is measured along the text x-axis, which a 90° matrix maps entirely onto device y, so `scale_x` is 0.
One caveat worth flagging. Word spaces cannot be recovered after the fact — whitespace-only items are dropped at extraction, and under the letter-spacing rotated labels use, a space advance is *smaller* than an ordinary letter advance, so no gap threshold separates them. Measured on one such header, the true word break is a 19.79pt gap while an ordinary letter gap reaches 15.77pt; on another, a word break is 10.95pt against an ordinary 11.11pt. Keeping the blanks for rotated matrices and letting the merge consume them is the only reliable route, and it costs a small predicate at the text-emitting sites.
Both directions assume 90° CCW, which is the assumption `correct_rotated_page` already encodes.
Measured over 30 technical PDFs:
| build | scrambled cells | documents affected |
|---|---|---|
| `585d36e` | 58 | 3 |
| with the pass | 0 | 0 |
Row-survival and column-consistency are unchanged, so this is not a trade. Nine documents additionally gain a corrected data row: a value that had been assigned to the wrong column lands in the right one once the header stops occupying it.
`cargo test` (full suite, unit + integration over `tests/fixtures/`) goes 975 → 978 with three regression tests added; no existing test changes. Can open a PR if useful.
## Environment
- pdf-inspector `585d36e`, version 0.1.7
- rustc 1.92.0 (ded5c06cf 2025-12-08)
- Linux 7.0.0
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in src/extractor/content_stream.rs at correct_rotated_page and trace the extraction path before line grouping and merge_text_items. Run make_repro_rotated.py, then cargo run --release --bin pdf2md on the generated PDF and cargo test. Done means the rotated headers read in the expected order without interleaving, while body rows remain unchanged and regression coverage passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100