Reproduce Training S3DIS
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 365
- Avg merge
- 4h 6m
- Merged PRs (30d)
- 1
Description
### Checklist
- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D-ML/issues).
- [X] I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).
### My Question
## Hey community,
First, thanks that you created this awesome repo in the first place and second, thanks that you share many pretrained models!!!
Now to my question. I am trying to reproduce the training on the S3DIS dataset, but I am not able to achieve the same training mIoU as listed in the modelZOO.
### Is someone out there who knows how the listed Semantic Segmentation results were achieved?
I am using ml3d from the DEV branch, since I ran into the known issue with the shape of the weight tensor #601.
Since the given [training script pipeline](https://github.com/isl-org/Open3D-ML/blob/master/scripts/run_pipeline.py) relies on the corrupted ```open3d.ml``` package, I had to rewrite my own ```pipeline.py``` to pull e.g. the ```SemSegLoss``` from the ml3d DEV repo. In my opinion, this should not make any difference. Training runs without any error.
I trained the two ```torch``` models: ```RandLANet``` and ```PointTransformer```.
Configurations were always used from the ropo's ```/ml3d/configs/``` presented ```.yaml``` files.
The only adjustment was to set the ```num_workers: 0``` option, to run on my machine.
```yaml
pipeline:
name: SemanticSegmentation
num_workers: 0
```
Final results after the mIoU converges is:
| model | max. val. mIoI | epochs| expectation|
|--------|-----------------|--------|---------------|
| PointTransformer | 59.4 | 1500| ~ 69.2 |
| RandLAnet | 50. | 1000 | ~ 70.9 |
Training is performed on a Docker Container, with the NVidia Image:
```Docker
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04
```
- Ubuntu 22.04
- PyTorch: 1.13 + cu116
- open3D: 0.17.0
Host system: Windows 10
GPU: one RTX 3090
Can someone explain the difference in training results?
I was expecting the config-files to be exactly the configuration which was used to reproduce the models that can be also downloaded.
Is the simple answer to my question just that this is not the case and the pretrained models rely on heavy Hyper Parameter Tuning, or is it exactly the case, and I am doing something wrong?
Thanks for your help,
cheers Lukas
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.