Unstructured-IO / Unstructured-IO/unstructured-inference

`(cid:x)` elements showing up in `unstructured` output

Open

#117 opened on May 19, 2023 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language: Python
Stars: 209
Forks: 73
PR merge metrics: No merged PRs in 30d

Description

Currently in this test we're getting (cid:0) where previously we didn't. Seems to slip through because the number of (cid:x) elements is below the cid_ratio. The goal of this issue is to take a look and see if there's anything we need to adjust related to the "needs_ocr" logic.

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Research direction

Start with test_unstructured/partition/test_auto.py around the referenced test at line 295, then inspect unstructured_inference/inference/elements.py around the cid_ratio and needs_ocr logic. Run the test to reproduce the (cid:0) output and determine whether the OCR decision should handle these elements differently. Done means the test no longer exposes unexpected (cid:x) elements and the relevant behavior is covered.

Written by the indexing model from the issue text.

Assessment

Tech stack: python
Domain: computer-vision
Issue type: Bug
Difficulty: 3/5
Estimated time: 1-2 days
Activity status: Stale
Clarity: Mostly clear
Newbie friendliness: 45/100

A short digest of beginner-friendly GitHub issues.