Slow inference with RandLANet on Kitti

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D-ML/issues).
- [X] I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

I'm running inference on Kitti to test the real-time detection capabilities of randlanet. However the .run_inference(data) step on Kitti data takes appx 4-6 seconds. I was under the impression that the inference should be well under one second per frame/sweep (the original paper states 0.04s). For now I'm just quick testing on a laptop with a rtx3000 gpu.

- Is this this what should be expected? I.e. are the timings/numbers from the paper not including passing data to/from gpu?
- Are the original settings in the randlanet_semantickitti.yml ok with what is done here?
- Is there any way of displaying real-time data in the visualizer from Open3d-ML or is it just for training/testing batches of images?

* https://openaccess.thecvf.com/content_CVPR_2020/papers/Hu_RandLA-Net_Efficient_Semantic_Segmentation_of_Large-Scale_Point_Clouds_CVPR_2020_paper.pdf

My code is attached below

```python
import os
import open3d
import open3d.ml as _ml3d
import open3d.ml.torch as ml3d
import numpy as np
import time

COLOR_MAP = {
0: (0., 0., 1.),
1: (0., 1., 0.),
2: (1., 0., 0.),
3: (1., 1., 0.),
4: (0., 1., 1.),
5: (1., 0., 1.),
6: (1., 1., 1.),
7: (0., 0., 0.5),
8: (0., 0.5, 0.),
9: (0.5, 0., 0.),
10: (0.5, 0.5, 0.),
11: (0., 0.5, 0.5),
12: (0.5, 0., 0.5),
13: (0.5, 0.5, 0.5),
14: (0., 0., 0.25),
15: (0., 0.25, 0.)
}

cfg_file = "/path-to/Open3D-ML/ml3d/configs/randlanet_semantickitti.yml"
cfg = _ml3d.utils.Config.load_from_file(cfg_file)

model = ml3d.models.RandLANet(**cfg.model)
cfg.dataset['dataset_path'] = "/path-to-kitti_dataset"
dataset = ml3d.datasets.SemanticKITTI(cfg.dataset.pop('dataset_path', None), **cfg.dataset)
pipeline = ml3d.pipelines.SemanticSegmentation(model, device='cuda', **cfg.pipeline)

## download the weights.
ckpt_folder = "./logs/"
os.makedirs(ckpt_folder, exist_ok=True)
ckpt_path = ckpt_folder + "randlanet_semantickitti_202201071330utc.pth"
randlanet_url = "https://storage.googleapis.com/open3d-releases/model-zoo/randlanet_semantickitti_202201071330utc.pth"
if not os.path.exists(ckpt_path):
cmd = "wget {} -O {}".format(randlanet_url, ckpt_path)
os.system(cmd)

## load the parameters.
pipeline.load_ckpt(ckpt_path=ckpt_path)

## create window
o3dvis = open3d.visualization.Visualizer()
o3dvis.create_window(window_name="Lidar PCL", height=1024, width=1024, left=250, top=50)
o3dvis.get_render_option().point_size = 1.0
o3dvis.get_render_option().background_color = np.asarray([0.121, 0.121, 0.121])
test_split = dataset.get_split("test")
testdata = open3d.geometry.PointCloud()
testdata.points = open3d.utility.Vector3dVector(test_split.get_data(0)["point"])
o3dvis.add_geometry(testdata)

inference_time = 0
num_it = 20
current_iteration = 0

for iteration in range(num_it):
print("Current iteration: ", current_iteration)
data = test_split.get_data(iteration)
start_inference = time.time()
result = pipeline.run_inference(data)
end_inference = time.time()
print("Current time: ", end_inference - start_inference)
if current_iteration > 0:
inference_time += end_inference - start_inference
testdata.points = open3d.utility.Vector3dVector(data['point'])
colors = [COLOR_MAP[clr % 15] for clr in list(result['predict_labels'])]
testdata.colors = open3d.utility.Vector3dVector(colors)
o3dvis.update_geometry(testdata)
o3dvis.update_renderer()
o3dvis.poll_events()
current_iteration += 1

inference_time /= (num_it - 1.0)
print("Average time: ", inference_time)

o3dvis.destroy_window()
```