google / google/cameratrapai

Multispecies classification

Open
#8 13 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
567
Forks
68
Avg merge
4h 39m
Merged PRs (30d)
5

Description

Many thanks getting this together! Works straight of the box.

I'm running into some problems with labeling images with multiple species. While the detection works great, the classification output only seems available for the species with the highest probability; classifier.batch_predict is predicting the entire image and not the individual detections. Is there a way to classify the individual detections?

For example, this input image has three species; but only mountain zebra is stored in the output json:
![Image](https://github.com/user-attachments/assets/9bf81c57-8f21-49f4-8407-bda219fab777)

```"classifications": {
"classes": [
"e178c377-987c-4c7d-ac56-6adcea8d75e1;mammalia;perissodactyla;equidae;equus;zebra;mountain zebra",
"3877192e-7bc5-4311-a737-0262ba9d3395;mammalia;perissodactyla;equidae;equus;grevyi;grevy's zebra",
"dd39bbd5-077c-482e-9d33-bd176116c870;mammalia;perissodactyla;equidae;equus;quagga;plains zebra",
"c134e0ab-cf96-45ec-bae6-60b94995f71b;mammalia;cetartiodactyla;bovidae;aepyceros;melampus;impala",
"c9b58a23-a776-4927-b737-68ed6c34fcba;mammalia;cetartiodactyla;bovidae;madoqua;guentheri;guenther's dik-dik"
],
"scores": [
0.9494189023971558,
0.02687123976647854,
0.014292134903371334,
0.000818404951132834,
0.00047174980863928795
]
```

Related; by default the output image only gets 'label' assigned (animal in this case).
![Image](https://github.com/user-attachments/assets/a622759f-8c99-4549-9825-90195320d27c)
It's quite straightforward to re-label the bboxes with the species name; but in the case of multiple species everything would be labeled as the highest-ranked species. This was just a quick and dirty relabeling; please note that the confidence scores correspond to the detection scores; not to the classification confidence score.

![Image](https://github.com/user-attachments/assets/ee375d4b-f4f5-4b90-be44-519b3d0ff9c2)

When cropping the image to just one species, everything works as expected.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.