isl-org / isl-org/Open3D-ML

Bug in semantic segmentation metric

Open
#485 0 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2.3k
Forks
365
Avg merge
4h 6m
Merged PRs (30d)
1

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D-ML/issues).
- [X] I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### Describe the issue

I found in semseg_metric.py. the function 'acc()' calculate the mean per-class acc at the end, not overall accuracy as the document describes.

In semantic_segmentation.py, it uses this same value for both 'mean acc' and 'overall acc'

### Steps to reproduce the bug

```python
'semantic_segmentation.py'
Line 638-641:
for key, val in acc_dicts[-1].items():
writer.add_scalar("{}/ Overall".format(key), val, epoch)
for key, val in iou_dicts[-1].items():
writer.add_scalar("{}/ Overall".format(key), val, epoch)

Line 645-646:
log.info(f"Mean acc train: {acc_dicts[-1]['Training accuracy']:.3f} "
f" eval: {acc_dicts[-1]['Validation accuracy']:.3f}")

'semseg_metric.py'
Line 54:
accs.append(np.nanmean(accs))
```

### Error message

_No response_

### Expected behavior

_No response_

### Open3D, Python and System information

```markdown
- Operating system: Ubuntu 18.04
- Python version: Python 3.8
- Open3D version: 0.14.1+caa3ae0a5
- System type: x86
- Is this remote workstation?: yes
- How did you install Open3D?: build from source
```

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.