aws-samples / aws-samples/amazon-textract-textractor
Incorrect order of text layouts due to compare_bounding_box() used in group_elements_horizontally()
- Dominant language
- Jupyter Notebook
- Stars
- 493
- Forks
- 163
- PR merge metrics
- No merged PRs in 30d
Description
When I send a PDF with the following paragraph (which is a bit tilted, part of [this PDF file](https://www.accessdata.fda.gov/cdrh_docs/pdf/P010032A.pdf))
and use `Document.get_text()`, I get the following text where the order of the lines are shuffled.
```
Administration (FDA) has completed its review of your premarket approval application the The Center for Devices and Radiological Health (CDRH) of the Food and Drug This device is indicated as an aid in the management of chronic intractable pain programmer, the Model 1232 programming wand and the Model 1210 patient magnet. of the following components: the Model 3608 pulse generator, the Model 3850 patient (PMA) for the Genesis Neurostimulation (IPG) System. The System includes trunk and/or limbs, including unilateral or bilateral pain associated with failed back that surgery the PMA is approved subject to the conditions described below and in the syndrome, intractable low back pain and leg pain. We are pleased to inform you "Conditions of Approval" (enclosed). You may begin commercial distribution of the device upon receipt of this letter.
```
I debugged the code, and it looks like it's due to `text_util.compare_bounding_box()`, which is called from `layout.group_elements_horizontally()`.
`group_elements_horizontally()` receives a list of elements, which are layout texts for this paragraph.
The first element has `BoundingBox` as `x: 0.08591524511575699, y: 0.4836207926273346, width: 0.6273355484008789, height: 0.03193599358201027` and `text` as `'The Center for Devices and Radiological Health (CDRH) of the Food and Drug'`.
The second element has `BoundingBox` as `x: 0.08505144715309143, y: 0.5002045631408691, width: 0.6902255415916443, height: 0.03553390130400658` and `text` as `'Administration (FDA) has completed its review of your premarket approval application the'`.
`group_elements_horizontally()` sorts the elements by using `compare_bounding_box()`, and due to the following block, `compare_bounding_box()` sorts the elements by x axis instead of y axis.
```
if abs(ay_mid - by_mid) < delta:
if a.bbox.x > b.bbox.x:
return 1
else:
return -1
```
Because of that, the second element comes before the second element after the sort.
`compare_bounding_box()` was introduced in [this commit](https://github.com/aws-samples/amazon-textract-textractor/commit/0919d085f12643e6debe34210edcaf8f97c142fa#diff-9b85e90b50abd17134d8b1cd2f013738f3afe791ec46d9c7237e4aa0f9afa2b7), but it's unclear to me what was the heuristic behind the logic.
Could you please improve / fix the logic of `compare_bounding_box()`, and/or add an option to not use the heuristic and simply order the elements by y axis?
Contributor guide
Assessment
This issue has not been assessed yet.