Pointcloud shifts position while trying to rotate

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

I have a pointcloud which I am visualizing using `SceneWidget()`. After loading it, I change the camera position using `self.widget3d.scene.camera.look_at()` to get my desired view. Now if I try to rotate the pointcloud using Left click + drag, the pointcloud initially shifts it's position to the left by a considerable amount and then continues rotating. I do not want it to shift. What can be done to avoid this?