Point cloud construction

Hey there,

My goal is to create separate point clouds out of pedestrians' masks, an image, and a depth. Before I've been masking both images and depth based on each mask, creating rgbd from them, and creating a point cloud in the end. But this is too expensive in terms of time.

This is an example of a point cloud I get with the abovementioned approach:


I've tried to write the projection myself using the exact formulas that were described in the docstring, but my point cloud is tilted i.e. every straight line is not really straight anymore. Is there something special when creating a point cloud from rgb in open3d that I'm missing?

```
def create_cloud(image, depth, intrinsics):
"""Creates a point cloud from the whole image and depth map
Args:
image (ndarray): color image
depth (ndarray): depth map
intrinsics (ndarray): intrinsic parameters
Returns:
pt_clouds (PointCloud): resulting point clouds
"""
fx, fy = intrinsics[0][0], intrinsics[1][1]
cx, cy = intrinsics[0][2], intrinsics[1][2]
image = (image - image.min()) / (image.max() - image.min())
u, v = np.where(depth != 0)
z = depth[depth != 0]
x = (u - cx) / fx * z
y = (v - cy) / fy * z
pts = np.vstack((y, x, z)).T
pt_cloud = o3d.geometry.PointCloud()
pt_cloud.points = o3d.utility.Vector3dVector(pts)
pt_cloud.colors = o3d.utility.Vector3dVector(image[u, v, ...])
return pt_cloud
```

Resulting point cloud:

Another option I've been considering is to create the general point cloud and then get points from it based on a mask, but it looks like it's not implemented yet.