facebookresearch / facebookresearch/sonata

Inaccruacy in sementic segmentation

Open
#23 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
798
Forks
56
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for the patience in answering the issues.
We are working on using VGGT cloud point outputs as input to Sonata, with indoor scenes. However, after using the transformation pipeline, there is a loss of information which impacts the segmentation accuracy:
```
config = [
dict(type="CenterShift", apply_z=True),
dict(
type="GridSample",
grid_size=0.02,
hash_type="fnv",
mode="train",
return_grid_coord=True,
return_inverse=True,
),
dict(type="NormalizeColor"),
dict(type="ToTensor"),
dict(
type="Collect",
keys=("coord", "grid_coord", "color", "inverse"),
feat_keys=("coord", "color", "normal"),
),
]
transform = sonata.transform.Compose(config)
```
We tried to reduce the grid_size=0.02 down to 0.001 which slightly improves accuracy. Would like to know if there is any other way to improve on the performance (eg: axis alignment issues?). By the way, one cause might be our sample cloud point does not contain predefined segmentation classes, is that a big problem, or there are ways around it?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the supplied Sonata transform.Compose pipeline on the VGGT indoor point-cloud output, comparing GridSample settings such as grid_size=0.02 and 0.001. Check whether the transformation or missing predefined segmentation classes explains the accuracy loss, and validate any finding with segmentation-accuracy measurements.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.