isl-org / isl-org/Open3D-ML

SemanticKitti/Randlanet/SemanticSegmentation: Prediction labels offset by -1?

Open
#670 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
2.3k
Forks
365
Avg merge
4h 6m
Merged PRs (30d)
1

Description

### Checklist

- [x] I have searched for [similar issues](https://github.com/isl-org/Open3D-ML/issues).
- [x] I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [x] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `main` branch).

### My Question

I'm training my model on a dataset containing labels `[4, 12, 14]`. But on inference I get back `[3, 11, 13]` with accuracies in the high 90%s. Checking internally, that's because the code doesn't include unlabeled:0 in its set of predictable labels (which makes sense) and we arrive at 19 possible labels indexed as 0-18 (not including unlabeled). As opposed to the original 20 indexed as 0-19 (including unlabeled). So I sort of get it. I just find the return values unintuitive, since they're not indexed the same as my input values. Rather offset by -1. Which - from the outside - you have to implicitly know.

The fix for the extrenal code is pretty trivial. Just increment by 1. But again, at least to me, this seems like an upstream issue.

Thanks for your help. :)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing label handling in SemanticKitti/RandLANet/SemanticSegmentation, focusing on how unlabeled:0 and the predictable labels are represented during inference. Compare the returned labels with the input values [4, 12, 14] and determine the expected public behavior. Done means the label mapping is consistent with the dataset values or the offset is clearly documented and covered by an appropriate test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.