3D pointcloud to 2D projection with a specific field of view

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

I am trying to project a 2D image from a 3D point cloud. Herein, the 3D point cloud is constructed using RGB and Depth images whose FOV is 120 degree. Once the 3D point cloud is constructed, I wanted to reproject a 2D image from it using a FOV 80 degree. The resolution of the images used for building 3D point cloud and the reprojected 2D image are same ```[width: 1820, height: 940]```.

I calculated the intrinsic matrix by following the below steps

The camera model used here is ```PinHole```.

Step 1: Calculating focal lengths fx and fy

```
hfov = fov / 360. * 2. * np.pi
fx = img_w / (2. * np.tan(hfov / 2.))

vfov = 2. * np.arctan(np.tan(hfov / 2) * img_h / img_w)
fy = img_h / (2. * np.tan(vfov / 2.))

* fx and fy are in pixel length

```

Step 2: Calculating image centers u0, v0

```
u0, v0 = img_w / 2, img_h / 2
```

Step 3: Form the matrix

```
[[fx, 0, u0, 0],
[0, fy, v0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]]
```

I used this intrinsic matrix and the extrinsic matrix (calculated from x,y,z, roll, ptich and yaw) to reproject the 2D image with 80 degree FOV using Open3D api ```pcd.project_to_rgbd_image``` from 3D point cloud which was constructed using 120 degree FOV.

**When I follow the above steps, I ended up with a 2D image which is zoomed in a lot than expected**. However, if I reproject the 2D image with 120 degree FOV then the resulting image is almost close to the expectation (some extra regions are projected but camera to plane distance is perfect).

You can find the sample output below

**Reprojected image with FOV 80 degree from the point cloud (ego vehicle will be missing because it is not part of the RGB-D images)**

[![Reprojected image with FOV 80 degree from the point cloud (ego vehicle will be missing because it is not part of the RGB-D images)][2]][2]

**Reference image (taken with same FOV (80 degree), resolution, and extrinsic parameters)**

[![Reference image (taken with same FOV (80 degree), resolution, and extrinsic parameters)][1]][1]

I am very certain that the zoom in effect comes from the intrinsic matrix.

I am assuming that I am missing or doing something wrong while calculating ```fx``` and ```fy```. Perhaps, there may be a need of adjustment factor while dealing with different FOV's or some inconsistencies in the units.

Any help here would be appreciated.

[1]: https://i.stack.imgur.com/5Xi3z.jpg
[2]: https://i.stack.imgur.com/duVa9.jpg