facebookresearch / facebookresearch/sonata
Inaccruacy in sementic segmentation
- Dominant language
- Python
- Stars
- 798
- Forks
- 56
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thanks for the patience in answering the issues.
We are working on using VGGT cloud point outputs as input to Sonata, with indoor scenes. However, after using the transformation pipeline, there is a loss of information which impacts the segmentation accuracy:
```
config = [
dict(type="CenterShift", apply_z=True),
dict(
type="GridSample",
grid_size=0.02,
hash_type="fnv",
mode="train",
return_grid_coord=True,
return_inverse=True,
),
dict(type="NormalizeColor"),
dict(type="ToTensor"),
dict(
type="Collect",
keys=("coord", "grid_coord", "color", "inverse"),
feat_keys=("coord", "color", "normal"),
),
]
transform = sonata.transform.Compose(config)
```
We tried to reduce the grid_size=0.02 down to 0.001 which slightly improves accuracy. Would like to know if there is any other way to improve on the performance (eg: axis alignment issues?). By the way, one cause might be our sample cloud point does not contain predefined segmentation classes, is that a big problem, or there are ways around it?
Contributor guide
Research direction
Start by reproducing the supplied Sonata transform.Compose pipeline on the VGGT indoor point-cloud output, comparing GridSample settings such as grid_size=0.02 and 0.001. Check whether the transformation or missing predefined segmentation classes explains the accuracy loss, and validate any finding with segmentation-accuracy measurements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100