aws-samples / aws-samples/amazon-textract-textractor
[textractprettyprinter] List contents are duplicated when generating text output using `get_text_from_layout_json`
- Dominant language
- Jupyter Notebook
- Stars
- 493
- Forks
- 163
- PR merge metrics
- No merged PRs in 30d
Description
# Current Behavior
While trying to create markdown or text files from AWS Textract JSON output using the `get_text_from_layout_json` function, the contents of **_ALL_** the list items are duplicated in the output.
# Expected Behavior
Each list item's contents should be included in the output only once.
# Related Issues
- #274
# Possible Solution
The [AWS docs on Textract Layout Response Objects](https://docs.aws.amazon.com/textract/latest/dg/layoutresponse.html) mention that in the case of `LAYOUT_LIST` elements, their children can point to `LAYOUT_TEXT` elements, which is the case here.
> Layout elements can also point to different objects, such as TABLE objects, Key-Value pairs, or **LAYOUT_TEXT elements in the case of LAYOUT_LIST**
Due to this, when getting all layouts from the Textract JSON output ([LinearizeLayout._get_layout_blocks](https://github.com/aws-samples/amazon-textract-textractor/blob/bf8febc0108976136491c62d5adf72613ed7ef5b/prettyprinter/textractprettyprinter/t_pretty_print_layout.py#L31)), the `LIST_LAYOUT` as well as its child `TEXT_LAYOUT` layout elements are included, which leads to the duplication in output text.
The [get_text_from_layout_json](https://github.com/aws-samples/amazon-textract-textractor/blob/bf8febc0108976136491c62d5adf72613ed7ef5b/prettyprinter/textractprettyprinter/t_pretty_print.py#L526) function is a wrapper over [LinearizeLayout.get_text](https://github.com/aws-samples/amazon-textract-textractor/blob/bf8febc0108976136491c62d5adf72613ed7ef5b/prettyprinter/textractprettyprinter/t_pretty_print_layout.py#L214-L227) function which loops over all layouts (blocks with `LAYOUT.*` type) from the Textract JSON output and collects the text contents from their children blocks.
The fix lies in the `LinearizeLayout._get_layout_blocks` function where we can exclude the `LAYOUT_TEXT` elements which are children of `LAYOUT_LIST` elements.
# Steps to Reproduce
Minimal reproduction repo: [adityachandak287/textractprettyprinter-list-duplication-bug-repro](https://github.com/adityachandak287/textractprettyprinter-list-duplication-bug-repro)
The repository contains the following for reference:
- [Sample PDF file](https://github.com/adityachandak287/textractprettyprinter-list-duplication-bug-repro/blob/71ea446013e09b9ccf4c0da8d52c954cb104c702/data/textract-test-document.pdf)
- [AWS Textract JSON output](https://github.com/adityachandak287/textractprettyprinter-list-duplication-bug-repro/blob/71ea446013e09b9ccf4c0da8d52c954cb104c702/data/textract-test-document.json)
- [Text output file](https://github.com/adityachandak287/textractprettyprinter-list-duplication-bug-repro/blob/71ea446013e09b9ccf4c0da8d52c954cb104c702/data/output-textract-test-document.txt) (showing the duplication of list items)
Environment
```
amazon-textract-caller==0.2.4
amazon-textract-prettyprinter==0.1.10
amazon-textract-response-parser==0.1.48
boto3==1.35.6
botocore==1.35.6
```
---
Edit: Added related issues section.
Contributor guide
Assessment
This issue has not been assessed yet.