Uninitialized UV0 corrupts legacy TriangleMesh rendering on Mesa Lavapipe
- Dominant language
- C++
- Stars
- 14k
- Forks
- 2.6k
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 6
Description
## Description
With the official Open3D 0.20.0 Python wheel on Ubuntu 26.04, `OffscreenRenderer` produces corrupted output when it uses Mesa Lavapipe software Vulkan. A small six-sided cylinder contains large white polygons that do not belong to the material or lighting.
The reproducer creates a cylinder, reconstructs the same mesh through the public `TriangleMesh(vertices, triangles)` constructor, recomputes its normals, and renders it. Passing `--control` renders the original factory-created mesh instead; that output is clean.
This does not require ambient occlusion, shadows, a ground plane, or application-specific geometry.
## Steps to reproduce
Install the official wheel and NumPy, save the script below as `repro.py`, then force Mesa Lavapipe:
```bash
python -m venv .venv
.venv/bin/pip install open3d==0.20.0 numpy
EGL_PLATFORM=surfaceless \
VK_DRIVER_FILES=/usr/share/vulkan/icd.d/lvp_icd.json \
.venv/bin/python repro.py
```
Run the clean control with:
```bash
EGL_PLATFORM=surfaceless \
VK_DRIVER_FILES=/usr/share/vulkan/icd.d/lvp_icd.json \
.venv/bin/python repro.py --control
```
```python
"""Open3D 0.20 software-Vulkan rendering corruption reproducer."""
import argparse
import numpy as np
import open3d as o3d
from open3d.visualization import rendering
parser = argparse.ArgumentParser()
parser.add_argument("--control", action="store_true")
args = parser.parse_args()
# This order is required for the reduced, allocation-sensitive reproduction.
renderer = rendering.OffscreenRenderer(1920, 1080)
renderer.scene.set_background([0.58, 0.58, 0.58, 1.0])
renderer.scene.show_skybox(False)
renderer.scene.view.set_post_processing(True)
renderer.scene.view.set_antialiasing(True)
renderer.scene.view.set_ambient_occlusion(False)
renderer.scene.view.set_shadowing(False, rendering.View.ShadowType.VSM)
mesh = o3d.geometry.TriangleMesh.create_cylinder(
radius=0.5, height=1.5, resolution=6, split=6
)
mesh.translate([0.0, 0.0, 0.75])
mesh.compute_vertex_normals()
# Rebuild the identical mesh through the public array constructor.
# --control skips this block and renders cleanly.
if not args.control:
mesh = o3d.geometry.TriangleMesh(
o3d.utility.Vector3dVector(np.asarray(mesh.vertices).copy()),
o3d.utility.Vector3iVector(np.asarray(mesh.triangles).copy()),
)
mesh.compute_vertex_normals()
material = rendering.MaterialRecord()
material.shader = "defaultLit"
material.base_color = [0.263, 0.117, 0.552, 1.0]
material.base_roughness = 0.55
renderer.scene.add_geometry("mesh", mesh, material)
renderer.scene.scene.set_sun_light(
[-0.4, -0.6, -1.0], [1.0, 1.0, 1.0], 75_000
)
renderer.scene.scene.enable_sun_light(True)
renderer.setup_camera(
45.0, [0.0, 0.0, 0.4], [2.3, -2.3, 1.8], [0.0, 0.0, 1.0]
)
pixels = np.asarray(renderer.render_to_image())
output = "control.png" if args.control else "corrupt.png"
o3d.io.write_image(output, o3d.geometry.Image(pixels), 9)
bright_pixels = int(np.all(pixels[:, :, :3] > 200, axis=2).sum())
print(f"Open3D {o3d.__version__}; wrote {output}; bright pixels: {bright_pixels}")
```
## Actual behavior
`corrupt.png` contains large, flat white polygons across the purple cylinder.
The relevant output is:
```text
[Open3D INFO] EngineInstance: Vulkan software device detected; using Filament's Vulkan backend
Open3D 0.20.0; wrote corrupt.png; bright pixels: 47600
FEngine resolved backend: Vulkan
Vulkan device driver: llvmpipe Mesa 26.0.8-1ubuntu0.3 (LLVM 21.1.8)
Selected physical device 'llvmpipe (LLVM 21.1.8, 256 bits)' from 1 physical devices.
```
I ran the reproducer three times on the same host. All three corrupted images were byte-for-byte identical:
```text
SHA256 ef247ff1ac757b9f5a6894723a8447eeba84ff50a01b9aec75b5da8334760d00
```
The control reports 50 bright pixels and does not contain the white polygons.
## Expected behavior
Reconstructing a `TriangleMesh` from copies of its vertex and triangle arrays and recomputing normals should render the same geometry without white polygons.
## Environment
- Open3D: 0.20.0 official Python wheel
- Python: 3.12.13
- OS: Ubuntu 26.04.1 LTS
- Kernel: 7.0.0-31-generic x86_64
- CPU: Intel Core Ultra 9 285K
- `mesa-vulkan-drivers`: 26.0.8-1ubuntu0.3
- `libvulkan1`: 1.4.341.0-1
- Vulkan device: llvmpipe, Mesa Lavapipe software Vulkan
- LLVM: 21.1.8
- Headless rendering; `EGL_PLATFORM=surfaceless`
## Likely cause in `TriangleMeshBuffers.cpp`
The legacy `TriangleMesh` path appears to upload an uninitialized UV attribute:
1. [`CreateColoredBuffers`](https://github.com/isl-org/Open3D/blob/main/cpp/open3d/visualization/rendering/filament/TriangleMeshBuffers.cpp#L189-L226) allocates `TexturedVertex` storage with `malloc`.
2. It writes position, tangent, and color for each vertex, but never writes `TexturedVertex::uv`. The struct's default member initializer does not run for storage obtained through `malloc`.
3. In [`ConstructBuffers`](https://github.com/isl-org/Open3D/blob/main/cpp/open3d/visualization/rendering/filament/TriangleMeshBuffers.cpp#L443-L473), the no-UV mesh takes the `CreateColoredBuffers` branch and then sets `has_uvs = true`.
4. [`BuildFilamentVertexBuffer`](https://github.com/isl-org/Open3D/blob/main/cpp/open3d/visualization/rendering/filament/TriangleMeshBuffers.cpp#L105-L139) consequently exposes `UV0` to Filament, pointing it at the unwritten bytes.
Two diagnostics support this explanation:
- Adding initialized zero UVs before `add_geometry` makes the output clean and byte-for-byte identical to the control:
```python
mesh.triangle_uvs = o3d.utility.Vector2dVector(
np.zeros((3 * len(mesh.triangles), 2))
)
```
- Changing glibc's heap fill through `MALLOC_PERTURB_` changes or removes the corruption. With the normal environment the render has 47,600 bright pixels; `MALLOC_PERTURB_=1` changes it to 15,356; values 42, 85, 170, and 254 produce the clean control image exactly.
A targeted fix may be to initialize `element.uv` in `CreateColoredBuffers`, for example:
```cpp
TexturedVertex& element = vertices[i];
element.uv = kDefault.uv;
```
This matches the fallback used by the legacy point-cloud buffer builder and the tensor triangle-mesh path. `has_uvs = true` should remain because Open3D's built-in lit and unlit materials require `UV0`.
## Additional observations
- Ambient occlusion is explicitly disabled in the reproducer.
- No ground plane or application-specific code is involved.
- Creating `OffscreenRenderer` before rebuilding the mesh is required for this reduced case. Rebuilding the mesh before creating the renderer rendered cleanly in my test.
- The symptom is topology and allocation sensitive. Some nearby cylinder resolutions render cleanly, while resolutions 6, 11, 12, 15, 18, 22, 24, 64, and 100 showed corruption on this host.
- The original, larger scene showed horizontal white bands. This reduction turns the symptom into white polygons on individual faces.
- This was found while testing the software-Vulkan path introduced by https://github.com/isl-org/Open3D/pull/7550.
I have prepared a focused Open3D patch that initializes the fallback UV and a software-Vulkan regression test using this conventional `TriangleMesh` case at 1920x1080. I can link the pull request here once submitted.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with CreateColoredBuffers and ConstructBuffers in cpp/open3d/visualization/rendering/filament/TriangleMeshBuffers.cpp, then trace how BuildFilamentVertexBuffer exposes UV0. Run repro.py with the Lavapipe environment and compare corrupt.png with control.png; done means the reconstructed mesh renders cleanly and the reported software-Vulkan regression test passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- computer-graphics, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100