How to get the 3D location of 2D Pixel in reconstructed Mesh?

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

I reconstructed a 3D Mesh from RGBD image using this function

```

def generate_mesh(image:np.ndarray, depth_image:np.ndarray, quality):
height, width, _ = image.shape

depth_image = (depth_image * 255 / np.max(depth_image)).astype('uint8')
image = (image * 255 / np.max(image)).astype('uint8')
# image = np.array(image)

# create rgbd image
depth_o3d = o3d.geometry.Image(depth_image)
image_o3d = o3d.geometry.Image(image)
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(image_o3d, depth_o3d,
convert_rgb_to_intensity=False)

# camera settings
camera_intrinsic = o3d.camera.PinholeCameraIntrinsic()
camera_intrinsic.set_intrinsics(width, height, 500, 500, width / 2, height / 2)

# create point cloud
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(rgbd_image, camera_intrinsic)

# outliers removal
cl, ind = pcd.remove_statistical_outlier(nb_neighbors=20, std_ratio=20.0)
pcd = pcd.select_by_index(ind)

# estimate normals
pcd.estimate_normals()
pcd.orient_normals_to_align_with_direction(orientation_reference=(0., 0., -1.))

# surface reconstruction
mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_poisson(pcd, depth=quality, n_threads=1)[0]

# rotate the mesh
rotation = mesh.get_rotation_matrix_from_xyz((np.pi, np.pi, 0))
mesh.rotate(rotation, center=(0, 0, 0))

# # save the mesh
# temp_name = next(tempfile._get_candidate_names()) + '.obj'
# o3d.io.write_triangle_mesh(temp_name, mesh)

return mesh
```

I want to segment a part of the image, and get the corresponding part from the mesh.
I came up with creating an image with 0 as value for all the part that I don't segment and non-zero for the part that I segmented

Something like this

I want to know where is those white-ish part exist in 3D mesh.

I thought I can get them by comparing the vertex color value, with the color in 2D image, but it seems like vertex_colors is being interpolated. I found some value looking like this (12,13,14) or (253,254,252). Even though my segmented image shoud have the same value for all the channel...

How do I get the location of those segmented parts in 3D coordinates?