microsoft / microsoft/TRELLIS

Distorted Meshes and Voxelization due to Inconsistent .ply Export

Open
#200 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.7k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

I've encountered an issue in the data generation pipeline where some meshes saved in the .ply format become distorted. This distortion subsequently leads to incorrectly generated voxelized shapes. The problem appears to stem from inconsistencies in how the original mesh is saved to the .ply file, causing certain parts of the shape to be misrepresented.

  • Some meshes, when exported to .ply and reloaded (e.g., with Trimesh), exhibit visible distortions. This is not a problem with the visualization, it occurs after reimporting the mesh.
  • These distortions propagate to the voxelization stage, resulting in inaccurate voxel representations.
  • The attached image visually demonstrates the issue. The top row shows the original pipeline's output, while the bottom row shows the improved output after applying the fix described below.
  • Tested on Blender Version: 3.2.2, Operating System: Ubuntu 22.04

Image

Proposed Solution (and Workaround):

I've found that applying a remeshing step before the triangulation and .ply export significantly improves the quality and consistency of the generated meshes and subsequent voxelizations. Specifically, I've implemented the following remeshing function and integrated it into the data generation pipeline before the 'convert_to_meshes()' and 'triangulate_meshes()' functions:

def combine_and_remesh() -> None:
    """Combines all mesh objects and applies a Remesh modifier.
    
    Returns:
        None
    """

    bpy.ops.object.select_all(action="DESELECT")
    objs = [obj for obj in bpy.context.scene.objects if obj.type == "MESH"]

    # Select all mesh objects
    for obj in objs:
        obj.select_set(True)

    # Join the selected mesh objects
    bpy.context.view_layer.objects.active = objs[0] #set active object so join works.
    bpy.ops.object.join()

    # Get the combined object
    combined_object = bpy.context.active_object

    # Remesh the combined object
    if combined_object and combined_object.type == 'MESH':
        remesh_modifier = combined_object.modifiers.new(name="Remesh", type='REMESH')
        # SMOOTH, BLOCKS, SHARP
        remesh_modifier.mode = 'SMOOTH'
        remesh_modifier.octree_depth = 9
        remesh_modifier.scale = 0.9
        remesh_modifier.use_remove_disconnected = False
        bpy.ops.object.modifier_apply(modifier="Remesh")
        print("Objects combined and remeshed.")

    bpy.ops.object.select_all(action="DESELECT")

This function does the following:

  1. Identifies and selects all objects of type "MESH" in the scene.
  2. Combines all selected meshes into a single mesh object.
  3. Applies a Remesh modifier: Uses Blender's built-in "Remesh" modifier with the SMOOTH mode to create a more uniform mesh. The key parameters are:
    • remesh_modifier.mode = 'SMOOTH' : Creates a smoother surface.
    • remesh_modifier.octree_depth = 9 : Controls the resolution of the remeshing (higher value = more detail, slower).
    • remesh_modifier.scale = 0.9 : A scaling factor (slightly less than 1 to help with internal intersections).
    • remesh_modifier.use_remove_disconnected = False: Keeps all parts of the mesh, even if they are not connected.
  4. Applies the modifier: Makes the remeshing permanent.
  5. Deselects all objects.

For improved performance, the following adjustments can be made without significantly impacting the quality of the final voxelized output. Reduce remesh_modifier.octree_depth = 8 speeds up the process. Setting remesh_modifier.scale = 0.75 can also contribute to faster computation. Using remesh mode BLOCKS instead of SMOOTH gives a mesh closer to the voxel shape, but also works.

Request:

I propose integrating a remeshing step, similar to the one described above, into the data generation pipeline. This should improve the robustness and accuracy of the mesh generation and subsequent voxelization. I believe the SMOOTH mode with adjusted parameters (or potentially the BLOCKS mode) offers a good balance between quality and performance. I'm open to suggestions on the best way to implement this within the existing codebase and would be happy to create a pull request.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace the data-generation pipeline around convert_to_meshes() and triangulate_meshes(), then inspect how mesh objects are exported to .ply and reloaded before voxelization. Compare the existing pipeline with the proposed remeshing step and validate the result on affected meshes in Blender 3.2.2. Done means reimported meshes are no longer distorted and their voxelized shapes are accurate.

Written by the indexing model from the issue text.

Assessment

Tech stack
blender, python
Domain
computer-graphics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.