docling-project / docling-project/docling

Not handling "fi" correctly occasionally

Open
#1,123 2 comments 0 reactions 1 assignee Claimed by @PeterStaar-IBM View on GitHub
bug pdf parsing
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
3d 4h
Merged PRs (30d)
98

Description

### Bug
Sometimes, the "fi" in the word "configuration" will be parsed as "GLYPH<222>", probably because the dot in the i is stuck with the f?

### Steps to reproduce
code:
```
from pathlib import Path
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.datamodel.base_models import InputFormat

from docling.datamodel.pipeline_options import PdfPipelineOptions, AcceleratorOptions
import torch
import time

@torch.no_grad()
def main():
t=time.time()
accelerator_options = AcceleratorOptions(num_threads=8)
accelerator_options.device = "cpu"
pipeline_options = PdfPipelineOptions()
pipeline_options.do_formula_enrichment = True
pipeline_options.accelerator_options = accelerator_options

converter = DocumentConverter(format_options={
InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)
})

result = converter.convert("example.pdf")
doc = result.document
doc.save_as_markdown(Path("mine.md"))
print(time.time()-t)

if __name__ == "__main__":
main()
```
example pdf file:
[A Branch-and-Price Algorithm for Multi-mode Resource Leveling.pdf](https://github.com/user-attachments/files/19100909/A.Branch-and-Price.Algorithm.for.Multi-mode.Resource.Leveling.pdf)

In the resulting `.md` file, it goes
> ## 3.2 Master Problem: A Model Based on Shift ConGLYPH<222>gurations
>
> In order to reduce the effects of 'losing the timing information' just described, we propose a model which exploits the problem structure by decomposing the time horizon. Based on the calendar for each resource type, every working shift represents a smaller subproblem for which sub-schedules are generated independently. These sub-schedules are linked by constraints ensuring that exactly one is chosen for each working shift. For each such sub-schedule, or conGLYPH<222>guration , we introduce a binary variable x ξ which indicates whether configuration ξ is chosen. We abbreviate j ∈ ξ to express that job j is executed in the shift corresponding to ξ . Every configuration ξ has an associated resource capacity R ξ and start times S jξ and completion times C jξ for each j ∈ ξ . Note that the mode of each job is determined by the start and completion times. The model reads:

The word "configuration" in bold or italic cannot be correctly parsed, even though they can be correctly copied and pasted.

### Docling version

Docling version: 2.25.0
Docling Core version: 2.21.1
Docling IBM Models version: 3.4.1
Docling Parse version: 3.4.0
Python: cpython-310 (3.10.16)
Platform: Linux-5.15.0-131-generic-x86_64-with-glibc2.35
### Python version

Python 3.10.16

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.