nans in the retrieved extrinsic and values flipped from what it was set to

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### My Question

When retrieving the extrinsic from the view control, after setting it, why do I get nans in the extrinsic and why is it flipped?

```
import numpy as np
import open3d
from open3d.visualization import ViewControl

np.set_printoptions(precision=2, suppress=True)

def main():
vis = open3d.visualization.Visualizer()
vis.create_window(visible=False)
view_control: ViewControl = vis.get_view_control()

vis.update_renderer()
vis.poll_events()

target = np.random.rand(3)

extrinsics = construct_extrinsics(target)

camera_parameters = view_control.convert_to_pinhole_camera_parameters()

extrinsics_out = camera_parameters.extrinsic
# no nans
print(extrinsics_out)

vis.update_renderer()
vis.poll_events()

camera_parameters.extrinsic = extrinsics
result = view_control.convert_from_pinhole_camera_parameters(camera_parameters, allow_arbitrary=True)

vis.update_renderer()
vis.poll_events()

assert result

# no nans
print(extrinsics)

camera_parameters = view_control.convert_to_pinhole_camera_parameters()

extrinsics_out = camera_parameters.extrinsic
# nans
print(extrinsics_out)

def construct_extrinsics(target: np.ndarray) -> np.ndarray:
eye = np.zeros(3)
forward = target - eye

assert np.allclose(np.linalg.norm(forward), np.linalg.norm(target))

forward /= np.linalg.norm(forward)

up = np.array([0.0, 1.0, 0.0])
right = np.cross(up, forward)
up = np.cross(right, forward)

up /= np.linalg.norm(up)
right /= np.linalg.norm(right)

assert up @ right < 1e-6
assert up @ forward < 1e-6
assert right @ forward < 1e-6

rotation = np.array([right, up, forward])
extrinsics = np.eye(4)
extrinsics[:3, :3] = rotation
extrinsics[:3, 3] = -(rotation @ eye)

return extrinsics

if __name__ == "__main__":
main()
```

```
# default
[[ 1. 0. 0. -0.]
[-0. -1. -0. 0.]
[-0. -0. -1. 0.]
[ 0. 0. 0. 1.]]

# my input
[[ 0.49 0. -0.87 -0. ]
[ 0.83 -0.31 0.47 -0. ]
[ 0.27 0.95 0.15 -0. ]
[ 0. 0. 0. 1. ]]

# the output (with nans and flipped entries)
[[-0.49 0. 0.87 nan]
[ 0.83 -0.31 0.47 nan]
[ 0.27 0.95 0.15 nan]
[ 0. 0. 0. 1. ]]
```

`open3d v0.18.0`