camelot-dev / camelot-dev/excalibur

Feature request: Option to Link PDF URL, refreshes each time download page is accessed

Open
#40 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.8k
Forks
238
PR merge metrics
No merged PRs in 30d

Description

I'd like to request an option to link PDFs (since PDF data often updates, it's much easier to keep data updated by linking than by manually uploading each time).

When a PDF is linked, and the rules have been set for that particular PDF, whenever the download page is accessed(example download page link: http://127.0.0.1:5000/jobs/3c90fc1b-a9d8-4d51-a83a-218d18d4893f), it automatically downloads from URL, then re-processes it with pre-defined rule, and displays the tables of the extracted data. This should work by just accessing the download page.

I can perhaps hire someone to get this done if you're willing to add it to the main project.

So steps are:
1. Link to PDF (example: https://www.lcfcu.org/home/fiFiles/static/documents/rates.pdf)
2. Set Rules for the PDF and save
3. Access the download page for that pdf (example: http://127.0.0.1:5000/jobs/3c90fc1b-a9d8-4d51-a83a-218d18d4893f)
4. Excalibur automatically fetches the PDF from link
5. Extracts data from PDF based on predefined rule
6. Displays like so:
![image](https://user-images.githubusercontent.com/19336285/50732138-390d1200-1129-11e9-9ec9-74f56265e658.png)

So in the future, whenever I detect the pdf has changed, I can access the download page link and it'll repeat the entire process again(steps 3-6).

Again, please let me know if you're open to have this change contribute to main source, if so, I can get it coded. I feel this change is extremely important since many pdf on web change frequently thus making this feature very useful.

Contributor guide

Open the contributing guide

Research direction

Start at the /jobs/ download page and trace the existing PDF upload and rule-processing flow; the issue does not name source files or tests. Done means a linked PDF is fetched when its download page is accessed, reprocessed with its saved rules, and the extracted tables are displayed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.