docling-project / docling-project/docling
PDF: URL char-by-char spacing + metadata field leakage in references
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
Two related text-spacing artifacts in PDF extraction:
**Artifact 1 — URL character spacing:** URLs in reference sections are rendered with single-character spacing, where each character is separated by a space:
```
h t t p s : / / d o i . o r g / 1 0 . 1 0 1 6 / j . p e q u i n . 2 0 2 3 . 1 2 . 0 0 2
```
**Artifact 2 — Metadata field leakage:** PDF metadata field names (WGROUP, STRING, PUBLICATION, PAGE, etc.) are extracted as spaced-character text and appear in the references section:
```
W G R O U P : S T R I N G : P U B L I C A T I O N
P A G E : S T R I N G : A R T I C L E / C H A P T E R
```
### Steps to reproduce
```python
from docling.document_converter import DocumentConverter
converter = DocumentConverter()
result = converter.convert("verkerk_forensic_necropsies_2026.pdf")
md = result.document.export_to_markdown()
# Artifact 1: URLs have character spacing
assert "h t t p s" in md # Bug: URL mangled
# Artifact 2: metadata fields leaked as text
assert "W G R O U P" in md # Bug: metadata leaked
```
### Expected behavior
1. URLs should be extracted as clean URLs: `https://doi.org/10.1016/...`
2. PDF metadata fields should not appear in the text output — they are structural, not content
### Source location
- **URL spacing**: `page_assemble_model.py:118 sanitize_text` — no URL-aware run un-spacing. The PDF producer stored URLs with explicit character spacing (one glyph per `Tj` operator), and the extractor normalized these to word breaks.
- **Metadata leakage**: `backend/docling_parse_backend.py` — the backend pulls all text from the page without discarding `/Artifact` marked-content or non-`Contents` `Tj` ops.
### Suggestion
1. For URLs: detect runs matching URL patterns (`https?://`, `doi.org`) and collapse inter-character spaces
2. For metadata: discard text runs matching known PDF metadata field patterns (single-char-spaced uppercase sequences like `Type`, `Subtype`, `Filter`, `WGROUP`, `STRING`)
### Docling version
2.115.0
### Regression test PDF
- `verkerk_forensic_necropsies_2026.pdf` (4.4 MB) — Springer journal article, both artifacts visible in the references section
Contributor guide
Research direction
Read page_assemble_model.py around sanitize_text and backend/docling_parse_backend.py, then run the supplied DocumentConverter reproduction with verkerk_forensic_necropsies_2026.pdf. Done means URLs export without character spacing and PDF metadata field names no longer appear in the extracted references text.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100