llmware-ai / llmware-ai/llmware

Issues with Table Extraction Accuracy in Sample

Open
#162 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
14.8k
Forks
2.9k
PR merge metrics
No merged PRs in 30d

Description

## Problem Description
When using the sample provided by the llmware project, I've encountered issues with the accuracy of table extractions. Specifically, not all tables are being extracted correctly.
As an example given in the sample, [Annual_Report_2003.pdf](https://github.com/llmware-ai/llmware/files/13594483/Annual_Report_2003.pdf).

## Steps to Reproduce
Run the sample extraction process as in [examples](https://github.com/llmware-ai/llmware/blob/main/examples/extract_pdf_tables.py). Set `query` parameter to be empty string.
Review the output and compare it to the expected tables within the documents. Only one table, to be exact, part of the table spreading from page 44-46, got correctly extracted.

## Expected Outcome
All tables within the sample documents should be identified and extracted accurately. In this file, tables that are supposed to be extracted are given below as screenshot,
Screenshot 2023-12-07 at 11 26 35 AM
Screenshot 2023-12-07 at 11 26 53 AM
Screenshot 2023-12-07 at 11 27 06 AM
Screenshot 2023-12-07 at 11 27 20 AM
Screenshot 2023-12-07 at 11 27 35 AM
Screenshot 2023-12-07 at 11 27 46 AM
Screenshot 2023-12-07 at 11 27 59 AM
Screenshot 2023-12-07 at 11 28 13 AM

## Actual Outcome
Only one csv outputs.
Please refer to the outcome [table_0.csv](https://github.com/llmware-ai/llmware/files/13594544/table_0.csv).
All other table contents, though extracted in json file, is labeled as text.

## Potential Impact
This issue may lead to incomplete or inaccurate data capture, which can affect the integrity of data analysis and further processing steps.

## Request for Assistance
I would appreciate any guidance on how to resolve this issue or any suggested workarounds. Additionally, if there are any plans to improve the table extraction feature in the near future, information on that would also be helpful.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/extract_pdf_tables.py, using Annual_Report_2003.pdf and an empty query as described. Compare the generated JSON and table_0.csv with the tables shown in the issue; done means all expected tables are identified and extracted as tables rather than labeled as text.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.