weecology / weecology/DeepForest
Should dataframes with empty predictions have a class recall of None or 0?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 774
- Forks
- 265
- PR merge metrics
- No merged PRs in 30d
Description
This may get superseded by the desire to transition to torchmetrics for evaluate.py, but I am having internal debate about how to format the evaluation dataframe for situations where the model returns no predictions.
related to https://github.com/weecology/DeepForest/issues/412
and uncovered during the PR here: https://github.com/weecology/DeepForest/pull/410
def test_evaluate_empty():
m = main.deepforest()
m.config["score_thresh"] = 0.8
csv_file = get_data("OSBS_029.csv")
root_dir = os.path.dirname(csv_file)
results = m.evaluate(csv_file, root_dir, iou_threshold = 0.4)
#Does this make reasonable predictions, we know the model works.
assert np.isnan(results["box_precision"])
assert results["box_recall"] == 0
df = pd.read_csv(csv_file)
assert results["results"].shape[0] == df.shape[0]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with evaluate.py and the test_evaluate_empty case using OSBS_029.csv. Review related issue 412 and pull request 410 to determine the expected representation for empty predictions and whether the existing precision, recall, and dataframe-shape assertions reflect it. Done means the behavior is decided and the relevant evaluation test passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, pandas, python
- Domain
- machine-learning, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100