Unable to control viewpoint with mouse when adding and removing geometries in visualizer

Hi there,

I am currently visualizing a segmented (ROI) point cloud from my realsense camera ( I am using the open3d realsense example for running it in real-time [(see here)](https://github.com/intel-isl/Open3D/blob/master/examples/python/misc/realsense.py)

I have a CAD model of a true object. ( I have a separate system to calculate the real objects pose== transformation matrix_CADtoRealife). In my code, I apply this transformation to the CAD PLY mesh every single time the object is in the image frame ( too_flag == TRUE) ... add_geometry, and update_geometry.... then I update the renderer, etc.

Since it is not guaranteed that the object will be in the next frame, I remove the geometry after that frame, so that it can be added, and updated at every single frame if present. ( End goal: only visualize the object when it is truly in frame)

When I do this, I can no longer manipulate the view orientation in the visualizer ( Mouse view control: via mouse click and move)... I can't seem to figure out if it is because I am visualizing a mesh or because I keep adding-removing, etc?

` if frame_count == 0:
vis.add_geometry(pcd)
vis.add_geometry(assemb_pcd)

#Update_geometry
vis.update_geometry(pcd)
vis.update_geometry(assemb_pcd)

if tool_flag == True:
vis.add_geometry(tool)
vis.update_geometry(tool)

#Render new frame
vis.poll_events()
vis.update_renderer()

if tool_flag==True:
vis.remove_geometry(tool)`