google-deepmind / google-deepmind/mujoco

Feature request: unbounded depth-tested line overlay in mjvScene

Open
#3,182 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
15.2k
Forks
1.8k
Avg merge
10d 16h
Merged PRs (30d)
25

Description

## Enhancement suggestion

**Current behavior:** Custom lines can only be added to a scene via `mjGEOM_LINE` geoms in the `scn->geoms[]` buffer, which shares the fixed `maxgeom` limit with all other visualization geometry. Applications that need many lines (LOD grid overlays, force vectors, debug annotations) quickly exhaust the geom budget.

**Desired behavior:** A separate, dynamically-growing line buffer in `mjvScene` with a simple push API:

```c
mjv_resetLines(&scn); // clear per frame
mjv_addLine(&scn, from, to, rgba); // add as many as needed
// ... mjr_render draws them with depth testing
```

Lines would be rendered by MuJoCo's OpenGL pipeline with proper depth testing, blending, and occlusion against scene geometry. No maxgeom limit.

**Why this is useful:**
- LOD wireframe overlays on height fields (water simulation, terrain)
- Force/velocity vector visualization at scale
- Debug annotations for robotics (contact normals, joint axes) without stealing geom slots
- Any application where line count is data-dependent and unbounded

**Proof of concept:** Working implementation at https://github.com/stanbot8/mujoco/tree/feature/unbounded-line-buffer

Changes (5 files, ~100 lines):
1. `mjvisualize.h`: Add `nline`, `maxline`, `linevert`, `linergba` to `mjvScene`
2. `mujoco.h`: Declare `mjv_addLine` and `mjv_resetLines`
3. `engine_vis_init.c`: Implement allocation, free, reset, and dynamic-grow add
4. `engine_vis_init.h`: Internal declarations
5. `render_gl3.c`: Render pass after transparent geoms, before labels, with depth test and blending

The buffer starts at 1024 lines, doubles on overflow, and is freed by `mjv_freeScene`. `mjv_resetLines` just sets `nline = 0` (no reallocation).

Tested with a multi-scale ocean simulation demo that draws LOD grid wireframes (coarse + fine + sub-cell) on a height field surface. Lines correctly occlude behind a floating sphere and the water surface itself.

Happy to add tests and follow the style guide if there is interest. This is independent of (but complementary to) issue #3181 (per-geom wireframe).

**Platform:** Windows 10, MuJoCo latest main branch

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.