google-deepmind / google-deepmind/mujoco

Mouse-click python api in passive viewer

Open
#2,574 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
15.2k
Forks
1.8k
Avg merge
10d 16h
Merged PRs (30d)
25

Description

### The feature, motivation and pitch

It would be nice to get access to the ID of the single/double-clicked (right or left mouse button) body/geom. Even better could be the ID of the body/geom of the mouse pointer location, and the mouse clicks accessed separately.

The key_callback of the passive viewer already allows for some nice interactivity with the simulation, but getting access to the mouse clicks would bring this to another level.

Some possible ways one could use this feature:

- Draw visualization markers only related to the clicked body/geom, allowing comprehensive data visualization without being too cluttered.
- Position a joint based on the position of the clicked object.
- Actuate the currently selected body.
- Highlighting of the body/geom that the mouse is hovering over.
- If there is also access to the 3D point on the surface of the clicked body/geom, that could be useful to set reference positions for a robot/actuator.

### Alternatives

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.