Different values for accuracy (validation and testing)
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 365
- Avg merge
- 4h 6m
- Merged PRs (30d)
- 1
Description
### Checklist
- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D-ML/issues).
- [X] I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).
### Describe the issue
I'm trying to train point transformer on a dataset based on S3DIS.
in the last epoch of the training process, I get a value for accuracy and miou as follows:
INFO - 2022-03-11 10:36:05,257 - semantic_segmentation - Mean acc train: 0.950 eval: 0.933
INFO - 2022-03-11 10:36:05,258 - semantic_segmentation - Mean IoU train: 0.901 eval: 0.902
But when I test my model on the same sample, I get new values for accuracy and miou:
Overall Testing Accuracy : 0.6765742520464817, mIoU : 0.6481393184649503
I'm using pipeline.run_test() to test my model.
I think the value of accuracy is being calculated differently in these stages.
Is there any solution to this problem?
Thank you!
### Steps to reproduce the bug
```python
model = ml3d.models.PointTransformer(**cfg.model)
pipeline = ml3d.pipelines.SemanticSegmentation(model=model, dataset=dataset, device="gpu", **cfg.pipeline)
pipeline.run_train()
# the following lines give the value of accuracy and MIoU in the last epoch:
pipeline.metric_val.acc()
pipeline.metric_val.iou()
# this line of code gives different values of accuracy and MIoU on the same sample of data
pipeline.run_test()
```
### Error message
_No response_
### Expected behavior
the value of acc and MIoU in the last epoch should be equal to the values coming from the function run_test since the same sample for validation and testing has been used.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.