docling-project / docling-project/docling-parse
Merged textline cells decode every run with one font, corrupting Symbol glyphs (→ becomes "fi")
- Dominant language
- C++
- Stars
- 333
- Forks
- 80
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 10
Description
### Summary
When adjacent text runs are merged into a single textline cell, every byte in that cell is decoded using one font — the one the cell reports as `font_key` — rather than the font each run was actually drawn in. A `/Symbol` run sharing a line with Helvetica text is therefore read through `Helvetica.afm`, where code 174 is `fi` instead of `arrowright`.
A source `Alpha → Bravo` comes out as `Alpha fi Bravo`. This is a quiet failure: the arrow becomes a plausible-looking *word*, so it survives review and silently degrades any downstream embedding or quoted citation, rather than surfacing as an obvious `GLYPH<…>` placeholder.
Runs that are **not** merged decode correctly, so this is not a missing Symbol encoding — `Symbol.afm` ships and is used correctly in the unmerged case.
### Versions
```
docling-parse == 7.8.0
docling == 2.113.0 (default DoclingParseV4 backend)
pypdf == 6.14.2
pypdfium2 == 5.12.0
Python 3.14.3, Windows
```
### Reproduction
```python
from docling_parse.pdf_parser import DoclingPdfParser
doc = DoclingPdfParser().load(path_or_stream="symbol-encoding-repro.pdf")
for _n, page in doc.iterate_pages():
for cell in page.textline_cells:
if "fi" in cell.text:
print(repr(cell.font_key), repr(cell.text))
```
**Actual**
```
'/F1' 'HotelHote fi Ind'
'/F1' 'Kilo fi Lim'
'/F1' 'NovemberNove fi Osc'
```
**Expected** — `fi` should be `→` (U+2192) in each.
`pypdf`, `pypdfium2` and docling's own `PyPdfiumDocumentBackend` all decode the same bytes correctly.
### What the PDF contains
A ReportLab-produced page that switches font mid-line to draw the arrow:
```
BT 1 0 0 1 176.0787 96.5 Tm (HotelHote ) Tj /F5 8.5 Tf 12 TL (®) Tj /F1 8.5 Tf 12 TL ( Ind) Tj T* ET
```
`/F5` is `/Symbol` with no `/Encoding` and no `/ToUnicode` — spec-legal, and it obliges the reader to use the font's built-in encoding. `/F1` is `/Helvetica` with `/Encoding /WinAnsiEncoding`.
### Analysis
The merged cell reports `font_key='/F1'`, and the byte drawn in `/F5` is resolved through `/F1`'s metrics. Comparing the shipped AFMs explains both glyphs that come out wrong:
| AFM | code 174 (0xAE) | code 72 (0x48) |
|---|---|---|
| `fonts/standard/Helvetica.afm` | `fi` (line 134) | `H` (line 66) |
| `fonts/standard/Symbol.afm` | `arrowright` (line 131) | — |
| `fonts/standard/ZapfDingbats.afm` | — | `a35`, a star (line 62) |
- The `/Symbol` arrow (174) read through Helvetica → `fi` ✔ matches observed output
- A `/ZapfDingbats` star (72) read through Helvetica → `H` ✔ same document, same cause
Note the AFM also wins over the `/WinAnsiEncoding` that `/F1` explicitly declares — under WinAnsi, 174 would be `®`, not `fi`. That places this in the same wrong-table-wins family as #187 (*MacRomanEncoding ignored when font matches known base font*), which points at `get_correct_character()` in `src/v2/pdf_resources/page_font.h` (lines 480–498). This report is a distinct case though: no declared encoding is being ignored here — the problem is that a merged cell applies one run's font to bytes belonging to another.
### It is merging-specific
A one-page PDF with the *same* font setup and the same `(®) Tj` in `/F5`, but whose runs stay in separate cells, decodes **correctly** to `→`. The encoding machinery is fine in isolation; only the merged path is affected.
### Suggested direction
Carry the per-run font through to character resolution so each byte is decoded with the font it was drawn in, rather than resolving a merged cell against a single `font_key`. Failing that, avoid merging runs across a font change when the fonts resolve characters differently.
symbol-encoding-repro.pdf (base64, 4865 bytes)
```
JVBERi0xLjQKJeLjz9MKMSAwIG9iago8PAovUHJvZHVjZXIgKHB5cGRmKQo+PgplbmRvYmoKMiAwIG9iago8PAovVHlwZSAvUGFnZXMKL0NvdW50IDEKL0tp
ZHMgWyA0IDAgUiBdCj4+CmVuZG9iagozIDAgb2JqCjw8Ci9UeXBlIC9DYXRhbG9nCi9QYWdlcyAyIDAgUgo+PgplbmRvYmoKNCAwIG9iago8PAovQ29udGVu
dHMgNSAwIFIKL01lZGlhQm94IFsgMCAwIDU5NS4yNzU2IDg0MS44ODk4IF0KL1Jlc291cmNlcyA8PAovRm9udCA2IDAgUgovUHJvY1NldCBbIC9QREYgL1Rl
eHQgL0ltYWdlQiAvSW1hZ2VDIC9JbWFnZUkgXQo+PgovUm90YXRlIDAKL1RyYW5zIDw8Cj4+Ci9UeXBlIC9QYWdlCi9QYXJlbnQgMiAwIFIKPj4KZW5kb2Jq
CjUgMCBvYmoKPDwKL0xlbmd0aCAzNTY1Cj4+CnN0cmVhbQoxIDAgMCAxIDAgMCBjbSAgQlQgL0YxIDEyIFRmIDE0LjQgVEwgRVQKcQpCVCAvRjEgNy41IFRm
IDkgVEwgRVQKLjQ2NjY2NyAuNDY2NjY3IC40NjY2NjcgcmcKQlQgMSAwIDAgMSA0NS4zNTQzMyAyNS41MTE4MSBUbSAoQWxwaGFBbHBoYUFscGhhQWxwaGFB
bHBoYUFscGhhQWxwaGFBbHBoYUFscGhhQWxwaGFBbHBoYUFscGgpIFRqIFQqIEVUCkJUIDEgMCAwIDEgNTI2LjE1MzggMjUuNTExODEgVG0gKEJyYXZvQikg
VGogVCogRVQKUQpxCjEgMCAwIDEgNTEuMzU0MzMgNzYyLjUzNTQgY20KcQowIDAgMCByZwpCVCAxIDAgMCAxIDAgMTguNSBUbSAvRjEgOS41IFRmIDE0IFRM
IChDaGFybGllQ2hhcmxpZUNoYXJsaWVDaGFybGllQ2hhcmxpZUNoYXJsaWVDaGFybGllQ2hhcmxpZUNoYXJsaWVDaGFybGllQ2hhcmxpZUNoYXJsaWVDaGFy
bGllQ2hhcmxpZUNoYXJsaWVDaGFyKSBUaiBUKiAoRGVsdGFEZWx0YURlbHRhRGVsdGFEZWx0KSBUaiBUKiBFVApRClEKcQoxIDAgMCAxIDcwLjg2NjE0IDYz
MC41MzU0IGNtCnEKLjkxNzY0NyAuOTE3NjQ3IC45MTc2NDcgcmcKbiAwIDEyNiA0NTMuNTQzMyAtMTggcmUgZioKMSAxIDEgcmcKbiAwIDEwOCA0NTMuNTQz
MyAtMTggcmUgZioKLjk3MjU0OSAuOTcyNTQ5IC45NzI1NDkgcmcKbiAwIDkwIDQ1My41NDMzIC0xOCByZSBmKgoxIDEgMSByZwpuIDAgNzIgNDUzLjU0MzMg
LTE4IHJlIGYqCi45NzI1NDkgLjk3MjU0OSAuOTcyNTQ5IHJnCm4gMCA1NCA0NTMuNTQzMyAtMTggcmUgZioKMSAxIDEgcmcKbiAwIDM2IDQ1My41NDMzIC0x
OCByZSBmKgouOTcyNTQ5IC45NzI1NDkgLjk3MjU0OSByZwpuIDAgMTggNDUzLjU0MzMgLTE4IHJlIGYqCjAgMCAwIHJnCkJUIC9GMiA4LjUgVGYgMTIgVEwg
RVQKQlQgMSAwIDAgMSA2IDExNC41IFRtIChFY2hvRWNoKSBUaiBUKiBFVApCVCAxIDAgMCAxIDE3Ni4wNzg3IDExNC41IFRtIChGb3h0cm90Rm94dHJvdCkg
VGogVCogRVQKQlQgL0YxIDguNSBUZiAxMiBUTCBFVApCVCAxIDAgMCAxIDYgOTYuNSBUbSAoR29sZkdvbCkgVGogVCogRVQKQlQgMSAwIDAgMSAxNzYuMDc4
NyA5Ni41IFRtIChIb3RlbEhvdGUgKSBUaiAvRjUgOC41IFRmIDEyIFRMIChcMjU2KSBUaiAvRjEgOC41IFRmIDEyIFRMICggSW5kKSBUaiBUKiBFVApCVCAx
IDAgMCAxIDYgNzguNSBUbSAoSnVsaWV0SnUpIFRqIFQqIEVUCkJUIDEgMCAwIDEgMTc2LjA3ODcgNzguNSBUbSAoS2lsbyApIFRqIC9GNSA4LjUgVGYgMTIg
VEwgKFwyNTYpIFRqIC9GMSA4LjUgVGYgMTIgVEwgKCBMaW0pIFRqIFQqIEVUCkJUIDEgMCAwIDEgNiA2MC41IFRtIChNaWtlTWlrZU1pa2VNaWtlTWkpIFRq
IFQqIEVUCkJUIDEgMCAwIDEgMTc2LjA3ODcgNjAuNSBUbSAoTm92ZW1iZXJOb3ZlICkgVGogL0Y1IDguNSBUZiAxMiBUTCAoXDI1NikgVGogL0YxIDguNSBU
ZiAxMiBUTCAoIE9zYykgVGogVCogRVQKQlQgMSAwIDAgMSA2IDQyLjUgVG0gKFBhcGFQYXBhUGFwYVBhcGFQYSkgVGogVCogRVQKQlQgMSAwIDAgMSAxNzYu
MDc4NyA0Mi41IFRtIChBbHBoYUFscGggKSBUaiAvRjUgOC41IFRmIDEyIFRMIChcMjU2KSBUaiAvRjEgOC41IFRmIDEyIFRMICggQnJhKSBUaiBUKiBFVApC
VCAxIDAgMCAxIDYgMjQuNSBUbSAoQ2hhcmxpZUNoYXJsaWVDaGFybGkpIFRqIFQqIEVUCkJUIDEgMCAwIDEgMTc2LjA3ODcgMjQuNSBUbSAoRGVsdCApIFRq
IC9GNSA4LjUgVGYgMTIgVEwgKFwyNTYpIFRqIC9GMSA4LjUgVGYgMTIgVEwgKCBFY2gpIFRqIFQqIEVUCkJUIDEgMCAwIDEgNiA2LjUgVG0gKEZveHRyb3RG
b3h0cm90RikgVGogVCogRVQKQlQgMSAwIDAgMSAxNzYuMDc4NyA2LjUgVG0gKEdvbGZHb2xmICkgVGogL0Y1IDguNSBUZiAxMiBUTCAoXDI1NikgVGogL0Yx
IDguNSBUZiAxMiBUTCAoIEhvdCkgVGogVCogRVQKcQoxIEoKMSBqCi43MzMzMzMgLjczMzMzMyAuNzMzMzMzIFJHCi4zIHcKbiAwIDEyNiBtIDQ1My41NDMz
IDEyNiBsIFMKbiAwIDAgbSA0NTMuNTQzMyAwIGwgUwpuIDAgMCBtIDAgMTI2IGwgUwpuIDQ1My41NDMzIDAgbSA0NTMuNTQzMyAxMjYgbCBTCm4gMCAxMDgg
bSA0NTMuNTQzMyAxMDggbCBTCm4gMCA5MCBtIDQ1My41NDMzIDkwIGwgUwpuIDAgNzIgbSA0NTMuNTQzMyA3MiBsIFMKbiAwIDU0IG0gNDUzLjU0MzMgNTQg
bCBTCm4gMCAzNiBtIDQ1My41NDMzIDM2IGwgUwpuIDAgMTggbSA0NTMuNTQzMyAxOCBsIFMKbiAxNzAuMDc4NyAwIG0gMTcwLjA3ODcgMTI2IGwgUwpRClEK
UQpxCjEgMCAwIDEgNTEuMzU0MzMgNjIyLjUzNTQgY20KUQpxCjEgMCAwIDEgNTEuMzU0MzMgNjAxLjUzNTQgY20KcQouNjY2NjY3IC42NjY2NjcgLjY2NjY2
NyBSRwouNSB3Cm4gMCAtOCA0OTIuNTY2OSAzMSByZSBTClEKcQpCVCAxIDAgMCAxIDggNSBUbSAxNSBUTCAvRjIgMTAgVGYgMCAwIDAgcmcgKEluZGlhSW5k
aWFJbmRpYUluZGlhSW5kaWFJbmRpYUkpIFRqIC9GMSAxMCBUZiAoIEp1bGlldEp1bGlldEp1bGlldEp1bGlldEp1bGlldEp1bGlldEp1bGlldEp1bGlldEp1
bGlldEp1bGllKSBUaiBUKiBFVApRClEKcQoxIDAgMCAxIDUxLjM1NDMzIDUzMS41MzU0IGNtCnEKLjY2NjY2NyAuNjY2NjY3IC42NjY2NjcgUkcKLjUgdwpu
IDAgLTggNDkyLjU2NjkgNzYgcmUgUwpRCnEKQlQgMSAwIDAgMSA4IDUwIFRtIDE1IFRMIC9GMiAxMCBUZiAwIDAgMCByZyAoS2lsb0tpbG9LaWxvS2lsb0tp
bG9LKSBUaiAvRjEgMTAgVGYgKCBMaW1hTGltYUxpbWFMaW1hTGltYUxpbWFMaW1hTGltYUxpbWFMaW1hTGltYUxpbWFMaW1hTGltYUxpbWFMaW1hTGltYUxp
bWFMaW1hTGltYUxpbWFMKSBUaiBUKiAoTWlrZU1pa2VNaWtlTWlrZU1pa2VNaWtlTWlrZU1pa2VNaWtlTWlrZU1pa2VNaWtlTWlrZU1pa2VNaWtlTWlrZU1p
a2VNaWtlTWlrZU1pa2VNaWtlTWlrZU1pa2UpIFRqIFQqIChOb3ZlbWJlck5vdmVtYmVyTm92ZW1iZXJOb3ZlbWJlck5vdmVtYmVyTm92ZW1iZXJOb3ZlbWJl
ck5vdmVtYmVyTm92ZW1iZXJOb3ZlbWJlck5vdmVtYmVyTm92ZW1iZSkgVGogVCogKE9zY2FyT3NjYXJPc2Nhck9zKSBUaiBUKiBFVApRClEKcQoxIDAgMCAx
IDUxLjM1NDMzIDQ5OC41MzU0IGNtCnEKMCAwIDAgcmcKQlQgMSAwIDAgMSAwIDQgVG0gL0YyIDE3IFRmIDIxIFRMIChQYXBhUGFwYVBhcGFQYXBhUGFwYVBh
cGFQYXBhUGFwYVBhcGFQYXBhKSBUaiBUKiBFVApRClEKcQoxIDAgMCAxIDUxLjM1NDMzIDQ0OC41MzU0IGNtCnEKMCAwIDAgcmcKQlQgMSAwIDAgMSAwIDMy
LjUgVG0gL0YxIDkuNSBUZiAxNCBUTCAoQWxwaGFBbHBoYUFscGhhQWxwaGFBbHBoYUFscGhhQWxwaGFBbHBoYUFscGhhQWxwaGFBbHBoYUFscGhhQWxwaGFB
bHBoYUFscGhhQWxwaGFBbHBoYUFscGhhQWxwaGFBbHBoYUFscGhhQWxwKSBUaiBUKiAoQnJhdm9CcmF2b0JyYXZvQnJhdm9CcmF2b0JyYXZvQnJhdm9CcmF2
b0JyYXZvQnJhdm9CcmF2b0JyYXZvQnJhdm9CcmF2b0JyYXZvQnJhdm9CcmF2b0JyYXZvQnJhdm9CcmF2b0JyYXZvKSBUaiBUKiAoQ2hhcmxpZUNoYXJsaWVD
aGFybGllQ2hhcmxpZUNoYXJsaWVDaGFybGllQ2hhcmxpZUNoYXJsaWVDaGFybGllQ2hhcmxpZUNoYXJsaWVDaGFybGllQ2hhcikgVGogVCogRVQKUQpRCiAK
CmVuZHN0cmVhbQplbmRvYmoKNiAwIG9iago8PAovRjEgNyAwIFIKL0YyIDggMCBSCi9GMyA5IDAgUgovRjQgMTAgMCBSCi9GNSAxMSAwIFIKPj4KZW5kb2Jq
CjcgMCBvYmoKPDwKL0Jhc2VGb250IC9IZWx2ZXRpY2EKL0VuY29kaW5nIC9XaW5BbnNpRW5jb2RpbmcKL05hbWUgL0YxCi9TdWJ0eXBlIC9UeXBlMQovVHlw
ZSAvRm9udAo+PgplbmRvYmoKOCAwIG9iago8PAovQmFzZUZvbnQgL0hlbHZldGljYS1Cb2xkCi9FbmNvZGluZyAvV2luQW5zaUVuY29kaW5nCi9OYW1lIC9G
MgovU3VidHlwZSAvVHlwZTEKL1R5cGUgL0ZvbnQKPj4KZW5kb2JqCjkgMCBvYmoKPDwKL0Jhc2VGb250IC9IZWx2ZXRpY2EtT2JsaXF1ZQovRW5jb2Rpbmcg
L1dpbkFuc2lFbmNvZGluZwovTmFtZSAvRjMKL1N1YnR5cGUgL1R5cGUxCi9UeXBlIC9Gb250Cj4+CmVuZG9iagoxMCAwIG9iago8PAovQmFzZUZvbnQgL1ph
cGZEaW5nYmF0cwovTmFtZSAvRjQKL1N1YnR5cGUgL1R5cGUxCi9UeXBlIC9Gb250Cj4+CmVuZG9iagoxMSAwIG9iago8PAovQmFzZUZvbnQgL1N5bWJvbAov
TmFtZSAvRjUKL1N1YnR5cGUgL1R5cGUxCi9UeXBlIC9Gb250Cj4+CmVuZG9iagp4cmVmCjAgMTIKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE1IDAw
MDAwIG4gCjAwMDAwMDAwNTQgMDAwMDAgbiAKMDAwMDAwMDExMyAwMDAwMCBuIAowMDAwMDAwMTYyIDAwMDAwIG4gCjAwMDAwMDAzNjEgMDAwMDAgbiAKMDAw
MDAwMzk3OCAwMDAwMCBuIAowMDAwMDA0MDUxIDAwMDAwIG4gCjAwMDAwMDQxNTggMDAwMDAgbiAKMDAwMDAwNDI3MCAwMDAwMCBuIAowMDAwMDA0Mzg1IDAw
MDAwIG4gCjAwMDAwMDQ0NjkgMDAwMDAgbiAKdHJhaWxlcgo8PAovU2l6ZSAxMgovUm9vdCAzIDAgUgovSW5mbyAxIDAgUgo+PgpzdGFydHhyZWYKNDU0Nwol
JUVPRgo=
```
Decode with:
```bash
base64 -d > symbol-encoding-repro.pdf # paste the block above, then Ctrl-D
```
Contributor guide
Research direction
Start with src/v2/pdf_resources/page_font.h, especially get_correct_character() around lines 480–498, then trace the merged textline-cell path that supplies its font. Reproduce with the supplied symbol-encoding-repro.pdf and Python snippet. Done means merged cells decode the Symbol arrow as → and the ZapfDingbats glyph correctly, while the existing unmerged case remains correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100