camelot-dev / camelot-dev/excalibur
Feature request: Option to Link PDF URL, refreshes each time download page is accessed
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 238
- PR merge metrics
- No merged PRs in 30d
Description
I'd like to request an option to link PDFs (since PDF data often updates, it's much easier to keep data updated by linking than by manually uploading each time).
When a PDF is linked, and the rules have been set for that particular PDF, whenever the download page is accessed(example download page link: http://127.0.0.1:5000/jobs/3c90fc1b-a9d8-4d51-a83a-218d18d4893f), it automatically downloads from URL, then re-processes it with pre-defined rule, and displays the tables of the extracted data. This should work by just accessing the download page.
I can perhaps hire someone to get this done if you're willing to add it to the main project.
So steps are:
1. Link to PDF (example: https://www.lcfcu.org/home/fiFiles/static/documents/rates.pdf)
2. Set Rules for the PDF and save
3. Access the download page for that pdf (example: http://127.0.0.1:5000/jobs/3c90fc1b-a9d8-4d51-a83a-218d18d4893f)
4. Excalibur automatically fetches the PDF from link
5. Extracts data from PDF based on predefined rule
6. Displays like so:

So in the future, whenever I detect the pdf has changed, I can access the download page link and it'll repeat the entire process again(steps 3-6).
Again, please let me know if you're open to have this change contribute to main source, if so, I can get it coded. I feel this change is extremely important since many pdf on web change frequently thus making this feature very useful.
Contributor guide
Research direction
Start at the /jobs/ download page and trace the existing PDF upload and rule-processing flow; the issue does not name source files or tests. Done means a linked PDF is fetched when its download page is accessed, reprocessed with its saved rules, and the extracted tables are displayed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100