Unstructured-IO / Unstructured-IO/unstructured
bug/incorrect text extraction by partition_pdf with hi_res strategy
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 15.5k
- Forks
- 1.3k
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 13
Description
Describe the bug
The text element is not exactly as written in pdf.
I have a pdf which consist tables. I am extracting elements for my RAG application with partition_pdf function - hi_res with yolox. It seems a simple text and it repeats in whole pdf but model seems miss one particular spot where actual text is "AUTOSAR Administration" and the element text returned by partition_pdf is "Teton".
To Reproduce
elements = partition_pdf(
filename="mypdf.pdf",
strategy="hi_res",
infer_table_structure=True,
model_name = "yolox"
)
for i, element in enumerate(elements):
print(f"\nElement {i+1}:")
print(f" Page Number: {element.metadata.page_number}")
print(f" Type: {type(element).name}")
print(f" Text: {element.text}")
if isinstance(element, Table):
print(f" This is a Table element. \n {element.metadata.text_as_html}")
elif isinstance(element, Title):
print(f" This is a Title element. Category Depth: {element.metadata.category_depth}")
elif isinstance(element, NarrativeText):
print(" This is a Narrative Text element.")
elif isinstance(element, ListItem):
print(" This is a List Item element.")
Expected behavior
Below should be the table html
| 2007-01-31 | | 2.1.0 | AUTOSAR Administration | e Harmonization of the document with other specifications (e.g. RTE) e Introduction of a new concept to support calibration and measurement - harmonized with RTE e Description of needs of the Software Component Template toward AUTOSAR services and of the interaction of the Software Component Template and |
| 2006-05-18 | | 2.0.0 | AUTOSAR Administration | Second |
| 2005-05-09 | | 1.0.0 | | AUTOSAR Administration | Initial release |
But instead it is as below (The "Teton" inside in the first line)
| 2007-01-31 | | 2.1.0 | Teton | e Harmonization of the document with other specifications (e.g. RTE) e Introduction of a new concept to support calibration and measurement - harmonized with RTE e Description of needs of the Software Component Template toward AUTOSAR services and of the interaction of the Software Component Template and |
| 2006-05-18 | | 2.0.0 | AUTOSAR Administration | Second |
| 2005-05-09 | | 1.0.0 | | AUTOSAR Administration | Initial release |
Screenshots
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the supplied partition_pdf example with strategy="hi_res", infer_table_structure=True, and model_name="yolox" against the reported PDF, then compare the generated table HTML with the expected output. Done means the first-row cell is extracted as "AUTOSAR Administration" rather than "Teton" without regressing the other rows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100