isl-org / isl-org/Open3D

How to use the raycasting together with the Visualizer?

Open
#5,076 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

#### Problem
I would like to render an image of a scene with the `Visualizer` class, detect a point of interest with opencv and later shoot a ray exactly at that position to determine the actual vertex. The ray should start at the camera's origin and be sent with respect to it's projection parameters.

To achieve correct results, the raycast scene and the visualizer scene have to have the exact same camera parameters. For now I would like to render the scene with orthographic projection, which would seem to be more precise.

#### Question
What is the best strategy to align both cameras and the two scenes? Is it possible to have a visualizer with the raycast scene attached?

#### Possible Strategy
As far as I understand, open3d already solved this problem within the [VisualizerWithVertexSelection](http://www.open3d.org/docs/latest/python_api/open3d.visualization.VisualizerWithVertexSelection.html) implementation, but it seems that in python the relevant method to select new points ([PickPoints()](https://github.com/isl-org/Open3D/blob/master/cpp/open3d/visualization/visualizer/VisualizerWithVertexSelection.cpp#L351)) is not available. Is there a reason for that?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.