docling-project / docling-project/docling-parse
`Invalid code point` when parsing some PDFs
- Dominant language
- C++
- Stars
- 333
- Forks
- 80
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 10
Description
I have encountered `Invalid code point` error while processing some of my PDFs. Sadly cannot share the actual PDF files or logs; sorry about this.
When the library tries to parse the CMAP for the `/ToUnicode` of specific font, some lines between `beginbfchar` and `endbfchar` are ignored because the code expects the key to be utf-8 character but they weren't. One of the case was ``.
Not sure if that is the cause, but in `std::string pdf_resource::get_string(uint32_t c)`, there is no mapping for the given `c(=57267)` in `cmap_numb_to_char`. As a result, the code proceeds to the `else if (32 <= c)` branch, and since `c` is not a valid UTF-8 value, passing it to `utf8::append` causes the `Invalid code point` error.
Contributor guide
Assessment
This issue has not been assessed yet.