aws-samples / aws-samples/amazon-textract-response-parser

Not able to extract Textract merge cell text properly

Open
#72 7 comments 0 reactions 0 assignees View on GitHub
python
Dominant language
TypeScript
Stars
236
Forks
97
PR merge metrics
No merged PRs in 30d

Description

Not able to extract the merge cell text properly. There is some issue with combine headers function. Textract not able to extract the top header text properly.

Reference:
t_doc = TDocumentSchema().load(textract_json)
ordered_doc = order_blocks_by_geo(t_doc)
trp_doc = Document(TDocumentSchema().dump(ordered_doc))
Now let’s iterate through the tables’ content, and extract the data into a DataFrame:

table_index = 1
dataframes = []
def combine_headers(top_h, bottom_h):
bottom_h[3] = top_h[2] + " " + bottom_h[3]
bottom_h[4] = top_h[2] + " " + bottom_h[4]
for page in trp_doc.pages:
for table in page.tables:
table_data = []
headers = table.get_header_field_names() #New Table method to retrieve header column names
if(len(headers)>0): #Let's retain the only table with headers
print("Statememt headers: "+ repr(headers))
top_header= headers[0]
bottom_header = headers[1]
combine_headers(top_header, bottom_header) #The statement has two headers. let's combine them
for r, row in enumerate(table.rows_without_header): #New Table attribute returning rows without headers
table_data.append([])
for c, cell in enumerate(row.cells):
table_data[r].append(cell.mergedText) #New Cell attribute returning merged cells common values
if len(table_data)>0:
df = pd.DataFrame(table_data, columns=bottom_header)

Document table format:
image

with above logic:
image

With small changes in the combine header, my issue got solved to some extent:

```
def combine_headers(top_h, bottom_h):
for i in range(len(top_h)):
if bottom_h[i] != top_h[i]:
bottom_h[i] = top_h[i] + ' ' + bottom_h[i]
else :
bottom_h[i] = bottom_h[i]
```

But there is some issue with textract top header detection,
image

Contributor guide

Open the contributing guide

Research direction

Start by tracing the table header path around get_header_field_names(), combine_headers(), and Cell.mergedText in the response parser. Reproduce the supplied Textract table example and compare the detected top and bottom headers with the screenshots. Done means merged-cell text and multi-row headers are extracted consistently, with regression coverage for the reported case.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python, typescript
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.