docling-project / docling-project/docling
implicit assumption all files are utf-8 encoded
Open
bug
triage/close-stale
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
`_guess_from_content` implicitly decodes bytes using utf-8, this is unfortunately not correct because there are many different encodings. Utf-16 is seen in the wild and cp1252 as well. It's not possible
### Steps to reproduce
Create a cp1252 encoded txt file and try to parse it with docling.
### Docling version
2.50.0
Contributor guide
Assessment
This issue has not been assessed yet.