docling-project / docling-project/docling-parse

`Invalid code point` when parsing some PDFs

Open
#145 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
333
Forks
80
Avg merge
1d 14h
Merged PRs (30d)
10

Description

I have encountered `Invalid code point` error while processing some of my PDFs. Sadly cannot share the actual PDF files or logs; sorry about this.

When the library tries to parse the CMAP for the `/ToUnicode` of specific font, some lines between `beginbfchar` and `endbfchar` are ignored because the code expects the key to be utf-8 character but they weren't. One of the case was ``.

Not sure if that is the cause, but in `std::string pdf_resource::get_string(uint32_t c)`, there is no mapping for the given `c(=57267)` in `cmap_numb_to_char`. As a result, the code proceeds to the `else if (32 <= c)` branch, and since `c` is not a valid UTF-8 value, passing it to `utf8::append` causes the `Invalid code point` error.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.