docling-project / docling-project/docling
Not handling "fi" correctly occasionally
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 98
Description
### Bug
Sometimes, the "fi" in the word "configuration" will be parsed as "GLYPH<222>", probably because the dot in the i is stuck with the f?
### Steps to reproduce
code:
```
from pathlib import Path
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions, AcceleratorOptions
import torch
import time
@torch.no_grad()
def main():
t=time.time()
accelerator_options = AcceleratorOptions(num_threads=8)
accelerator_options.device = "cpu"
pipeline_options = PdfPipelineOptions()
pipeline_options.do_formula_enrichment = True
pipeline_options.accelerator_options = accelerator_options
converter = DocumentConverter(format_options={
InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)
})
result = converter.convert("example.pdf")
doc = result.document
doc.save_as_markdown(Path("mine.md"))
print(time.time()-t)
if __name__ == "__main__":
main()
```
example pdf file:
[A Branch-and-Price Algorithm for Multi-mode Resource Leveling.pdf](https://github.com/user-attachments/files/19100909/A.Branch-and-Price.Algorithm.for.Multi-mode.Resource.Leveling.pdf)
In the resulting `.md` file, it goes
> ## 3.2 Master Problem: A Model Based on Shift ConGLYPH<222>gurations
>
> In order to reduce the effects of 'losing the timing information' just described, we propose a model which exploits the problem structure by decomposing the time horizon. Based on the calendar for each resource type, every working shift represents a smaller subproblem for which sub-schedules are generated independently. These sub-schedules are linked by constraints ensuring that exactly one is chosen for each working shift. For each such sub-schedule, or conGLYPH<222>guration , we introduce a binary variable x ξ which indicates whether configuration ξ is chosen. We abbreviate j ∈ ξ to express that job j is executed in the shift corresponding to ξ . Every configuration ξ has an associated resource capacity R ξ and start times S jξ and completion times C jξ for each j ∈ ξ . Note that the mode of each job is determined by the start and completion times. The model reads:
The word "configuration" in bold or italic cannot be correctly parsed, even though they can be correctly copied and pasted.
### Docling version
Docling version: 2.25.0
Docling Core version: 2.21.1
Docling IBM Models version: 3.4.1
Docling Parse version: 3.4.0
Python: cpython-310 (3.10.16)
Platform: Linux-5.15.0-131-generic-x86_64-with-glibc2.35
### Python version
Python 3.10.16
Contributor guide
Assessment
This issue has not been assessed yet.