How is the pretrained PointRCNN model supposed to be used? + Training Issues PointRCNN

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D-ML/issues).
- [X] I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

Hi,

I am using the following versions:
- tensorflow: 2.5.3
- open3d: 0.15.2 (from pip), as well as a local git clone of the current master branch

I am currently trying to get PointRCNN Object Detection (TF) to work. For that I downloaded the pretrained weights and trying to validate the model using the following command:
```bash
python scripts/run_pipeline.py tf -c ml3d/configs/pointrcnn_kitti.yml --dataset.dataset_path --model.ckpt_path --split test --mode RCNN/RPN
```
I tried both modes. When in RPN mode, cls loss is at around 1.1 and reg loss at around 7.7. However, validation_MAPs are *always* 0.0 in all cases.
In RCNN mode, the validation losses are around 0.0 and the validation_map's are again 0.0 in all cases.

So what mode has the pretrained PointRCNN been trained on? How should it be used?

Also, when I try to train PointRCNN by myself (RPN), the same issue with always zero validation_MAPs exists. Even after 50+ epochs, they stay at 0.0 in all cases.
See the plots of the training and validation of the RPN below. Remember, this is with default settings.