camelot-dev / camelot-dev/camelot
All data goes into single row
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 546
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 3
Description
**Describe the bug**
The rows are detected, but all of the data is inserted into the first row for some PDFs. I'd like to separate the data into their own rows.
**Steps to reproduce the bug**
Steps used to install `camelot`:
1. `pip install "camelot-py[cv]"`
2. apt install ghostscript python3-tk
Steps to reproduce the behavior:
1. Trying to extract tables from the pdf below using the code below.
**Expected behavior**
The tables need to be extracted into separate rows.
**Code**
Add the Camelot code snippet that you used.
```
import camelot
tables = camelot.read_pdf("20-10-2020.pdf", pages="24-26", strip_text="*", split_text=True)
print(tables[0].df)
```
**PDF**
Add the PDF file that you want to extract tables from.
[20-10-2020.pdf](https://github.com/camelot-dev/camelot/files/6565708/20-10-2020.pdf)
**Screenshots**
Here is the PDF:

It seems like the rows are detected:

But all the data is in one row separated by a `' \n'`

**Environment**
- OS: Linux-4.19.128-microsoft-standard-x86_64-with-glibc2.29
- Python version: 3.8.2 [GCC 9.3.0]
- Numpy version: 1.20.3
- OpenCV version: 4.5.2.52
- Ghostscript version: 9.50
- Camelot version: 0.8.2
**Additional context**
Thank you for this wonderful tool. I have actually scrapped quite a lot COVID Mortality data from the same type of PDF using camelot, and it's awesome! No other tool is as accurate as this.
In our hospital, they were entering this data manually at first!! You've saved countless man hours!
Most of the time it works great out of the box. It has this issue only with some files. I'm attaching other files that have this issue too:
[21-10-2020.pdf](https://github.com/camelot-dev/camelot/files/6565718/21-10-2020.pdf)
[22-10-2020.pdf](https://github.com/camelot-dev/camelot/files/6565719/22-10-2020.pdf)
[24-10-2020.pdf](https://github.com/camelot-dev/camelot/files/6565720/24-10-2020.pdf)
[26-10-2020.pdf](https://github.com/camelot-dev/camelot/files/6565721/26-10-2020.pdf)
[29-10-2020.pdf](https://github.com/camelot-dev/camelot/files/6565722/29-10-2020.pdf)
[30-10-2020.pdf](https://github.com/camelot-dev/camelot/files/6565723/30-10-2020.pdf)
[31-10-2020.pdf](https://github.com/camelot-dev/camelot/files/6565724/31-10-2020.pdf)
Please let me know if I can add configuration options to change this behaviour. Thank you.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the issue with camelot.read_pdf on 20-10-2020.pdf, using pages 24-26, strip_text="*", and split_text=True, then compare the returned tables[0].df with the attached screenshots. Trace how detected rows and newline-separated cell data are handled; done means the affected PDFs produce separate rows without breaking the files that already work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100