camelot-dev / camelot-dev/camelot

Negative value as accuracy of table.

Open
#44 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.8k
Forks
546
Avg merge
3d 17h
Merged PRs (30d)
3

Description

While testing I have faced a case where `table.accuracy` is negative number.

PDF:[page-3.pdf](https://github.com/camelot-dev/camelot/files/3455388/page-3.pdf)
Code:
```
tables=camelot.read_pdf('/Users/skatipomu/Table_Extraction_Camelot/page3.pdf',pages="all)
[table.accuracy for table in tables]
```
Output:
`[99.99999999999997, -20.852716930856104]`

I think the reason is because in `compute_accuracy` method in utils.py while calculating accuracy we are subtracting error percentage from 1. It is supposed to be in the range [0.0,1.0] but the errors passed on to this method contains error percentages in the range[0 to 100] which inturn is from `get_table_index` method. So dividing this error by 100 solved the issue for me.

```
def compute_accuracy(error_weights):
"""Calculates a score based on weights assigned to various
parameters and their error percentages.

Parameters
----------
error_weights : list
Two-dimensional list of the form [[p1, e1], [p2, e2], ...]
where pn is the weight assigned to list of errors en.
Sum of pn should be equal to 100.

Returns
-------
score : float

"""
SCORE_VAL = 100
try:
score = 0
if sum([ew[0] for ew in error_weights]) != SCORE_VAL:
raise ValueError("Sum of weights should be equal to 100.")
for ew in error_weights:
weight = ew[0] / len(ew[1])
for error_percentage in ew[1]:
**score += weight * (1 - error_percentage)**
except ZeroDivisionError:
score = 0
return score
```
from `score += weight * (1 - error_percentage)` to `score += weight * (1 - error_percentage/100.0)`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in utils.py at compute_accuracy and trace the error values supplied by get_table_index, then reproduce the negative result with page-3.pdf. Done means the reported table accuracy no longer becomes negative and the calculation matches the documented score range.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.