Why is Open3D's viewer posed slightly incorrectly?

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

I am trying to use Open3D to compare a pointcloud created from SfM with the original source image. I have previously done this in the following way:

Project each point in the point cloud using the camera `pose`, `K`, and distortion coefficients (`k1, k2, k3, k4, p1, p2`). The result of this projection is `projected_points_distorted.png` which is extremely close to `distorted_image.png` (the original image). Shown here:


I tried to replicate this using Open3D in the following way. I load the pointcloud in the viewer and then use the camera `pose` and `K` to position the viewer and capture a screen shot. The result is `projected_points_undistorted.png` which I compare to the `undistorted_image.png`. It should also be extremely close by if you flick between the two you will notice a shift. Shown here:


Here is a snippet showing the issue:

```
import open3d as o3d
import numpy as np

Config.width = 4864
Config.height = 3648
Config.fx = 3673.950036
Config.fy = 3673.950036
Config.cx = 2444.540736
Config.cy = 1826.73794
Config.k1 = 0.008617136609
Config.k2 = -0.023179458
Config.k3 = 0.02365399367
Config.p1 = -0.001470996524
Config.p2 = 0.0008450379141

Config.pose = np.array([
[0.03832601301, 0.9992395675, 0.007169616542, -19.35739055],
[0.9062510729, -0.03173479204, -0.4215470269, 0.2371150269],
[-0.4209989426, 0.02265368952, -0.9067781982, 1.02465339],
[0, 0, 0, 1]
])

Config.K = np.array([
[Config.fx, 0, Config.cx],
[ 0, Config.fy, Config.cy],
[ 0, 0, 1],
])

def build__pose(E, width, height):
I = o3d.camera.PinholeCameraIntrinsic()
I.set_intrinsics(width, height, Config.fx, Config.fy, Config.cx, Config.cy)
P = o3d.camera.PinholeCameraParameters()
P.intrinsic = I
P.extrinsic = E
return P

pcd = o3d.io.read_point_cloud("points.ply") #see attached zip
vis = o3d.visualization.Visualizer()
vis.create_window(window_name="Open3D", width=Config.width, height=Config.height, visible=False)
opt = vis.get_render_option()
opt.point_size = 1.5
opt.background_color = np.asarray([0, 0, 0])
vis.add_geometry(pcd)
vis.update_renderer()
pose = build__pose(Config.pose, width=Config.width, height=Config.height)
vis.get_view_control().convert_from_pinhole_camera_parameters(pose, allow_arbitrary=True)
vis.poll_events()
vis.update_renderer()
vis.capture_screen_image('projected_points_undistorted.png')
```
and for the image:

```
distorts = np.array([
Config.k1,
Config.k2,
Config.p1,
Config.p2,
Config.k3,
Config.k4,
0,
0,
])

img = cv2.imread('distorted_image.png')
ung = cv2.undistort(img, Config.K, distorts)
cv2.imwrite('undistorted_image.png', ung)
```

Does anyone know where this error is being introduced in Open3D or if there is a better way to produce an accurate view of a pointcloud using the viewer? I have attached all 4 images and the pointcloud here: https://www.dropbox.com/s/u9h977smf6ozjfv/Open3DError.zip?dl=0