aws-samples / aws-samples/amazon-textract-textractor

Incorrect order of text layouts due to compare_bounding_box() used in group_elements_horizontally()

Open
#389 3 comments 0 reactions 1 assignee Claimed by @Belval View on GitHub
need repro
Dominant language
Jupyter Notebook
Stars
493
Forks
163
PR merge metrics
No merged PRs in 30d

Description

When I send a PDF with the following paragraph (which is a bit tilted, part of [this PDF file](https://www.accessdata.fda.gov/cdrh_docs/pdf/P010032A.pdf))
image
and use `Document.get_text()`, I get the following text where the order of the lines are shuffled.
```
Administration (FDA) has completed its review of your premarket approval application the The Center for Devices and Radiological Health (CDRH) of the Food and Drug This device is indicated as an aid in the management of chronic intractable pain programmer, the Model 1232 programming wand and the Model 1210 patient magnet. of the following components: the Model 3608 pulse generator, the Model 3850 patient (PMA) for the Genesis Neurostimulation (IPG) System. The System includes trunk and/or limbs, including unilateral or bilateral pain associated with failed back that surgery the PMA is approved subject to the conditions described below and in the syndrome, intractable low back pain and leg pain. We are pleased to inform you "Conditions of Approval" (enclosed). You may begin commercial distribution of the device upon receipt of this letter.
```

I debugged the code, and it looks like it's due to `text_util.compare_bounding_box()`, which is called from `layout.group_elements_horizontally()`.

`group_elements_horizontally()` receives a list of elements, which are layout texts for this paragraph.

The first element has `BoundingBox` as `x: 0.08591524511575699, y: 0.4836207926273346, width: 0.6273355484008789, height: 0.03193599358201027` and `text` as `'The Center for Devices and Radiological Health (CDRH) of the Food and Drug'`.

The second element has `BoundingBox` as `x: 0.08505144715309143, y: 0.5002045631408691, width: 0.6902255415916443, height: 0.03553390130400658` and `text` as `'Administration (FDA) has completed its review of your premarket approval application the'`.

`group_elements_horizontally()` sorts the elements by using `compare_bounding_box()`, and due to the following block, `compare_bounding_box()` sorts the elements by x axis instead of y axis.
```
if abs(ay_mid - by_mid) < delta:
if a.bbox.x > b.bbox.x:
return 1
else:
return -1
```
Because of that, the second element comes before the second element after the sort.

`compare_bounding_box()` was introduced in [this commit](https://github.com/aws-samples/amazon-textract-textractor/commit/0919d085f12643e6debe34210edcaf8f97c142fa#diff-9b85e90b50abd17134d8b1cd2f013738f3afe791ec46d9c7237e4aa0f9afa2b7), but it's unclear to me what was the heuristic behind the logic.

Could you please improve / fix the logic of `compare_bounding_box()`, and/or add an option to not use the heuristic and simply order the elements by y axis?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.