llmware-ai / llmware-ai/llmware

Not extracting any tables!

Open
#950 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
14.8k
Forks
2.9k
PR merge metrics
No merged PRs in 30d

Description

```
import os
from llmware.library import Library
from llmware.retrieval import Query

def extract_pdf_tables(library_name):

print(f"\nExample: Parsing PDF Documents and Extracting Tables")

# Step 1 - create library
lib = Library().create_new_library(library_name)

# Step 2 - pull sample files
sample_files_path = "./pdfs"

# Step 3 - parse and extract all of the content from the PDF Documents
parsing_output = lib.add_files(input_folder_path=sample_files_path)

# Review the parsing output summary info - all of the text and table blocks are in Mongo collection
print("Update: parsing_output - ", parsing_output)

# Step 4 - export all of the content into .jsonl files with metadata
output_fp = "./output_csv"
print(f"Update: Step 4 - exporting all blocks into file path - {output_fp}")

output1 = lib.export_library_to_jsonl_file(output_fp, f"{library_name}_export")

# Step 5 - export all of the tables as csv with ''" in the query
print(f"Update: Step 5 - exporting all tables with into file path - {output_fp}")
output2 = Query(lib).export_all_tables(query="Topline", output_fp=output_fp)

return output2

if __name__ == "__main__":
extract_pdf_tables("pdf_table_lib_example")
```
- I have tried different query values ["politicians ", "elections", "Response"].

```
Update: parsing_output - {'docs_added': 1, 'blocks_added': 16, 'images_added': 1, 'pages_added': 3, 'tables_added': 0, 'rejected_files': []}
```
I have attached pdf and json file.

[survey.pdf](https://github.com/user-attachments/files/16537205/survey.pdf)
[pdf_table_lib_example_export.json](https://github.com/user-attachments/files/16537220/pdf_table_lib_example_export.json)

Thanks.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the shown extract_pdf_tables example, especially Library.add_files and Query(lib).export_all_tables, then compare the attached survey.pdf with the exported JSON. Check why parsing reports tables_added: 0 and whether the extracted content contains table blocks. Done means the sample PDF produces table records and export_all_tables writes the expected table output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.