Regression: pages with extractable text now return empty markdown (bisected to #231)

## Summary

On `main`, `extract_pages_markdown` returns an empty string for pages that
contain ordinary, correctly decoded text. The same pages extract fine on the
released version `0.2.6`.

The pages are classified as needing OCR by the `vector_text` signal, which #231
newly consulted from `extract_pages_markdown`, and that classification causes
the page markdown to be discarded. The
pages are ordinary tax forms: no images, decodable fonts, hundreds of
extractable characters. What trips the heuristic is the dotted rules of empty
form fields, drawn as thousands of sub-point rectangles.

On `0.2.6` the same pages are not flagged (`needs_ocr = false`) and their text
is returned.

Bisected to `371de80` (#231, *"fix: extract_pages_markdown's needs_ocr now
agrees with classify_pdf"*). Reverting that commit on top of `main` restores
non-empty markdown on all 11 pages, with the same per-page Cyrillic-letter
counts as `0.2.6`.

## Affected versions

| | result |
|---|---|
| `0.2.6` (PyPI) | text extracted |
| `main` @ `69039f2` | empty markdown, `needs_ocr = true` |

## Reproduction

Four publicly available PDFs published by the Russian Federal Tax Service.
Direct links and checksums:

| file | sha256 | source |
|---|---|---|
| `15403282pril1.pdf` | `9ae4fe979739bdfd85e475f060ff8a74d1371373287c3263cdfd8de57c9a5de1` | [card](https://www.nalog.gov.ru/rn77/about_fts/docs/15403282/) · [pdf](https://www.nalog.gov.ru/html/sites/www.new.nalog.ru/files/about_fts/docs/15403282pril1.pdf) |
| `15482941pril1.pdf` | `a851bf3a2944726cee9365d6b45375dfe485c192ff9932d78aed5b06b1919f55` | [card](https://www.nalog.gov.ru/rn77/about_fts/docs/15482941/) · [pdf](https://www.nalog.gov.ru/html/sites/www.new.nalog.ru/files/about_fts/docs/15482941pril1.pdf) |
| `16172374_1.pdf` | `87f0d5a15aa51876bcaa4d02925740f1f6783100e34703d418c6005c86b42e9b` | [card](https://www.nalog.gov.ru/rn77/about_fts/docs/16172374/) · [pdf](https://www.nalog.gov.ru/html/sites/www.new.nalog.ru/files/about_fts/docs/16172374_1.pdf) |
| `16571439_1.pdf` | `501e4568b72a5f45e2c9d669298b1eacce7cde72433181c1d1d3a6a18bc9dc09` | [card](https://www.nalog.gov.ru/rn77/about_fts/docs/16571439/) · [pdf](https://www.nalog.gov.ru/html/sites/www.new.nalog.ru/files/about_fts/docs/16571439_1.pdf) |

```python
from pdf_inspector import extract_pages_markdown

CASES = {
"15403282pril1.pdf": [7],
"15482941pril1.pdf": [25, 31, 40],
"16172374_1.pdf": [3, 4, 5],
"16571439_1.pdf": [3, 4, 5, 7],
}

for name, pages in CASES.items():
result = extract_pages_markdown(name)
for number in pages: # 1-based page numbers
page = result.pages[number - 1]
print(name, number, page.needs_ocr, len(page.markdown or ""))
```

**Expected** (and what `0.2.6` produces): `needs_ocr = False` and non-empty
markdown on all 11 pages.

**Actual** on `main`: `needs_ocr = True` and `markdown = ""` on all 11 pages.

## Evidence that the text layer is fine

Letters recovered per page (letters whose Unicode name contains `CYRILLIC`),
`0.2.6` vs `main`:

| file | page | 0.2.6 | main |
|---|---:|---:|---:|
| `15403282pril1.pdf` | 7 | 601 | 0 |
| `15482941pril1.pdf` | 25 | 191 | 0 |
| `15482941pril1.pdf` | 31 | 456 | 0 |
| `15482941pril1.pdf` | 40 | 209 | 0 |
| `16172374_1.pdf` | 3 | 119 | 0 |
| `16172374_1.pdf` | 4 | 91 | 0 |
| `16172374_1.pdf` | 5 | 166 | 0 |
| `16571439_1.pdf` | 3 | 165 | 0 |
| `16571439_1.pdf` | 4 | 127 | 0 |
| `16571439_1.pdf` | 5 | 128 | 0 |
| `16571439_1.pdf` | 7 | 247 | 0 |

**The clearest check is within a single build.** On `main`, for the very same
document and page, `extract_text_with_positions` returns the text while
`extract_pages_markdown` returns nothing:

```
main @ 69039f2, 16172374_1.pdf
page 3 extract_text_with_positions: 119 Cyrillic letters extract_pages_markdown: "" (needs_ocr=True)
page 4 extract_text_with_positions: 91 Cyrillic letters extract_pages_markdown: "" (needs_ocr=True)
page 5 extract_text_with_positions: 166 Cyrillic letters extract_pages_markdown: "" (needs_ocr=True)
```

On these pages the extracted text items contain **no `U+FFFD` replacement
characters, no Private Use Area characters, no C1 control characters
(`U+0080`–`U+009F`) and no `(cid:N)` tokens** — zero of each. So the text layer
decodes cleanly and only the markdown path is affected.

An independent extractor (`pypdfium2`) confirms that substantive text is present
on all 11 pages. We are not claiming its output is identical to `0.2.6` — only
that the pages are not blank and not garbled.

## Bisect

```
git bisect start 69039f2 a15ec2d
1d134e2 good
fabbb63 bad
ede4809 good
585d36e bad
371de80 bad <- first bad commit
```

`371de80b148f0cf41abf83ea238484dc20ad980b` — *fix: extract_pages_markdown's
needs_ocr now agrees with classify_pdf (#231)*.

Reverting it on top of `69039f2` restores all 11 pages to the exact `0.2.6`
letter counts.

## Notes on the mechanism

Three separable observations:

1. **It is the new `vector_text` signal, and it misreads dotted form fields.**
We instrumented `page_ocr_signals` and traced every contributor to
`needs_ocr`. On all 11 pages the result is the same: `vector_text = 1`,
`template_image = 0`, and the pre-existing text-quality path silent
(`has_text_quality_issue = false` — the page is absent from
`analyze_text_quality().pages_needing_ocr`). Your existing quality analysis
is **not** involved.

The pages trip all three conditions — `path_ops >= 1000`,
`path_ops > text_ops * 200`, `unique_alphanum < 30`:

| file, page | path_ops / text_ops / unique_alphanum | images | rectangles < 100 pt² |
|---|---:|---:|---:|
| `15403282pril1.pdf`, 7 | 20 010 / 37 / 27 | 0 | 10 000 |
| `15482941pril1.pdf`, 31 | 22 056 / 73 / 29 | 0 | 11 024 |
| `16172374_1.pdf`, 4 | 31 880 / 13 / 24 | 0 | 15 936 |
| `16571439_1.pdf`, 3 | 31 112 / 33 / 25 | 0 | 15 552 |

(Four rows shown; all 11 follow the same pattern.)

Those path operators are not outlined glyphs. Measured independently with
`pdfplumber`, the rectangles have a median size of **0.567 × 0.567 pt**
(median area 0.3215 pt²) on every one of the 11 pages, and they form the
**dotted rules of empty form fields**. We rendered a representative page of
each of the four documents to confirm this visually. Meanwhile the pages
carry 139–708 extracted characters and `decodable_fonts = 1`.

In short: a form whose empty fields are drawn as thousands of dots produces
a very high path-operator count, which is exactly the profile the heuristic
treats as vector-outlined text.

2. **The guard meant to prevent this is ASCII-only, and that is why it does not
fire.** The third condition is not incidental — your comment states its
purpose directly: *"Pages with real selectable text plus decorative paths
(column borders, dividers) have many unique alphanum chars — these are NOT
vector-outlined text."* That is precisely the kind of page we are reporting,
so the guard is aimed at exactly this case and simply misses it.

`unique_alphanum_chars` counts distinct **ASCII** alphanumeric bytes among
the raw string operands. On a Cyrillic page the body text contributes none
of them: what is counted is only the stray digits and Latin characters in
the form's boilerplate. All 11 pages land at 24–29 against a threshold of
`< 30`:

| file, page | text_ops | unique_alphanum | decodable_fonts |
|---|---:|---:|---:|
| `15403282pril1.pdf`, 7 | 37 | 27 | 1 |
| `15482941pril1.pdf`, 25 | 31 | 26 | 1 |
| `15482941pril1.pdf`, 31 | 73 | 29 | 1 |
| `15482941pril1.pdf`, 40 | 39 | 26 | 1 |
| `16172374_1.pdf`, 3 | 15 | 25 | 1 |
| `16172374_1.pdf`, 4 | 13 | 24 | 1 |
| `16172374_1.pdf`, 5 | 15 | 29 | 1 |
| `16571439_1.pdf`, 3 | 33 | 25 | 1 |
| `16571439_1.pdf`, 4 | 17 | 24 | 1 |
| `16571439_1.pdf`, 5 | 33 | 24 | 1 |
| `16571439_1.pdf`, 7 | 31 | 28 | 1 |

**You already handle this confound elsewhere in the same file.** Both
`alphanum_ok` and `alphanum_low` exempt pages that have decodable fonts and
enough text operators, with the comment *"CID-encoded fonts with ToUnicode
produce low unique_alphanum_chars in raw bytes but are fully decodable"*.
That exemption is not applied to `has_vector_text`. Every one of the 11
pages has `decodable_fonts = 1` and `text_ops` between 13 and 73, so the
exemption you already trust would clear all of them.

3. **The response is all-or-nothing at page granularity.** When the flag is
raised the page's markdown is replaced with an empty string. Even where the
signal is justified, callers lose the whole page rather than the affected
part, and cannot distinguish a genuinely blank page from a suppressed one
except via `needs_ocr`.

**Scope of our evidence, stated plainly.** Across a public corpus of 62 Russian
documents / 1364 pages, `vector_text` fires on exactly these 11 pages and
nowhere else. That means we have no *additional* sample of false positives
beyond the ones reported here — we are not claiming the heuristic is broadly
wrong, only that on every page where we have seen it fire, it was wrong.

We understand the intent of #231 — not presenting untrustworthy extraction as
trustworthy is the right goal, and the previous disagreement between
`classify_pdf` and `extract_pages_markdown` was a real inconsistency. This
report is only about the cases where the page is in fact fine.

## Happy to send a patch

The smallest change we can see is to give `has_vector_text` the same exemption
the two other call sites already apply — a predicate you already compute and
already rely on:

```rust
let has_vector_text = path_ops >= 1000
&& path_ops > text_ops.saturating_mul(200)
&& unique_alphanum_chars < 30
&& !(has_decodable_text_fonts && text_ops >= 10);
```

That clears all 11 pages reported here without loosening any threshold.
Widening `unique_alphanum_chars` beyond ASCII would also work and is arguably
more faithful to the comment's intent, but it changes the meaning of a field
several branches read, so we would not propose it uninvited.

If a PR would help, we would send it with a **minimized synthetic fixture** —
real text plus thousands of sub-point rectangles — rather than vendoring these
tax PDFs into your test suite, and we would keep your existing genuine
vector-outlined-text fixture green, since the point is to narrow the heuristic
and not to disarm it. The public documents above would stay as an external
reproduction.

If you would rather fix it yourselves, that is fine too and we will happily
re-test and report back.

## Environment

```
pdf-inspector 0.2.6 (PyPI) and main @ 69039f2, built locally
rustc 1.97.1
platform Windows 11, x86_64
python 3.13
```

All measurements were taken at `69039f2`. At the time of writing `main` is at
`f4b8c9e`; the only commit between them touches `SECURITY.md`, and
`src/detector.rs` and `src/lib.rs` are byte-identical across the two.

## Related issues in this tracker

- **#227** is the report that led to #231. This is not a duplicate of it: #227
is about pages whose classification disagreed, this is about what the fix
now does to pages that were fine.
- **#252** reports the same visible symptom — `needs_ocr = true` together with
empty markdown — from a different trigger, a plain text page with no vector
content. Ours requires `path_ops >= 1000`, so the two cannot share a cause;
what they do share is the page-level all-or-nothing response noted above.
- **#319** and its open PR **#339** concern pages with *no painted content at
all*. That is the opposite input from these pages, which are dense with
painted content, so a fix there would not cover this case.

Start in src/detector.rs at has_vector_text and compare its conditions with the existing decodable-font exemptions used by alphanum_ok and alphanum_low. Reproduce the case with a minimized fixture containing decodable text and many small rectangles, then exercise extract_pages_markdown. Done means the fixture retains markdown and needs_ocr is false while the existing genuine vector-text fixture remains covered.

Written by the indexing model from the issue text.