isl-org / isl-org/Open3D

Offscreen mesh rendering initializes Gaussian-splat GLX sharing context without DISPLAY

Open
#7,554 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [x] I searched for similar issues.
- [x] I tested the official Linux development wheel first.
- [x] I reproduced the target path from source at the exact reported revision.

### Revised description

Ubuntu testing narrows this report. A valid surfaceless ordinary-mesh render does reach the optional Gaussian-splat GLX sharing-context initialization, and that helper emits a misleading warning because no X display exists. It does **not**, however, obstruct the ordinary Filament mesh render in the tested build: the render completes and the PNG is correct. The useful fix is therefore to skip optional Gaussian-splat interop initialization for explicitly surfaceless EGL runs, eliminating the failed GLX probe and warning. This is not evidence of a mesh-rendering failure.

### Environment and revisions

```text
Ubuntu 26.04.1 LTS, x86_64
Linux 7.0.0-31-generic
Python 3.12.13
Open3D 1a9eb990f9a20936c30c428568c602bdef760744
Filament v1.54.0, tag commit c1a3450d9c0a64c329a17bff4f3a7f68a2474214
Mesa 26.0.8 / EGL 1.7.0
DISPLAY unset
EGL_PLATFORM=surfaceless
LIBGL_ALWAYS_SOFTWARE=true
```

The first test used the official unmodified Linux development wheel `open3d-0.19.0+1a9eb99-cp312-cp312-manylinux_2_35_x86_64.whl`, SHA-256 `e3368125a937fb4438276612ee46b35d56cbf4a944e4c899a2e83bacb7ac5303`. That wheel currently contains no `open3d/resources/` entries and fails before graphics initialization:

```text
RuntimeError: [Open3D Error] ... Could not find resource file /tmp/open3d-upstream-installed-e3368125/open3d//resources
Exit status: 1
```

That packaging failure is separate from this issue, so I continued with an exact-revision source build.

### Source-build prerequisite

The upstream Filament v1.54.0 Linux build is GLX-only. `EGL_PLATFORM=surfaceless` cannot change that at runtime. For both sides of this A/B test I used the same separately recorded prerequisites: compile the GLX and EGL backends, select `PlatformEGLHeadless` when `EGL_PLATFORM=surfaceless`, use a pbuffer-capable EGL config, guard a null `glGetString(GL_EXTENSIONS)` result for core desktop GL, and bind desktop OpenGL on the compiler worker. Those changes only make the requested EGL-headless path runnable; the A/B difference below is solely Open3D's optional splat initialization.

### Exact example and command

```python
import open3d as o3d

renderer = o3d.visualization.rendering.OffscreenRenderer(320, 240)
mesh = o3d.geometry.TriangleMesh.create_sphere(0.1)
mesh.compute_vertex_normals()
material = o3d.visualization.rendering.MaterialRecord()
material.shader = "defaultLit"
renderer.scene.add_geometry("sphere", mesh, material)
renderer.setup_camera(60, [0, 0, 0], [0.3, -0.4, 0.2], [0, 0, 1])
o3d.io.write_image("sphere.png", renderer.render_to_image())
```

```bash
env -u DISPLAY EGL_PLATFORM=surfaceless LIBGL_ALWAYS_SOFTWARE=true \
python -u -X faulthandler mesh_capture.py
```

### Baseline: optional splat initialization enabled

```text
[Open3D WARNING] GLFWWindowSystem: X11 GLFW init failed (XWayland not available?). Falling back to headless mode.
[Open3D WARNING] GaussianSplatOpenGLContext: GS helper window failed. Gaussian Splat rendering is not available.
FEngine resolved backend: OpenGL
render_exit=0
sphere.png: PNG image data, 320 x 240, 8-bit/color RGB, non-interlaced
6cbe939f192110ed63ec0aba0e7418f2c9f249d2fe21255b1c56ea2954aab2aa sphere.png
Exit status: 0
```

This warning establishes that execution reached `GaussianSplatOpenGLContext::Initialize()`. There is no failure backtrace to report: initialization is recoverable and the application exits successfully.

Baseline wheel SHA-256: `cd696521e01aa7762cba2e572588fac712f667c41a6f14b014756901991ef9ef`.

### Isolated control: skip optional splat contexts for surfaceless EGL

The only Open3D change was to detect `EGL_PLATFORM=surfaceless` in `EngineInstance::EngineInstance()` and avoid both optional Gaussian-splat Vulkan and GL interop initialization in that case. Filament engine creation remains unchanged.

```text
[Open3D WARNING] GLFWWindowSystem: X11 GLFW init failed (XWayland not available?). Falling back to headless mode.
FEngine resolved backend: OpenGL
render_exit=0
sphere.png: PNG image data, 320 x 240, 8-bit/color RGB, non-interlaced
6cbe939f192110ed63ec0aba0e7418f2c9f249d2fe21255b1c56ea2954aab2aa sphere.png
Exit status: 0
```

The splat-specific warning disappears and the output is byte-identical. Gaussian splats remain available in display-backed runs.

### Result

Confirmed as redundant/noisy optional initialization, not as an ordinary-mesh obstruction. The isolated guard is behavior-preserving for this mesh case and suppresses the unavailable GLX-helper probe under explicit surfaceless EGL.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at EngineInstance::EngineInstance(), where optional Gaussian-splat Vulkan and GL interop initialization is performed, and reproduce the provided surfaceless EGL command. Compare the baseline and guarded runs: an ordinary mesh render should complete with identical PNG output, without the splat-specific warning under EGL_PLATFORM=surfaceless, while display-backed runs retain splat initialization.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
computer-graphics
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.