Cannot make deform_as_rigid_as_possible method work on other meshes besides Armadillo.ply
- Dominant language
- C++
- Stars
- 14k
- Forks
- 2.6k
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 6
Description
**Issue Description**
Hello, I'm trying to deform a mesh by using the as rigid as possible (ARAP) method as shown in [this example](http://www.open3d.org/docs/latest/tutorial/Advanced/mesh_deformation.html).
The method works fine on the Armadillo mesh, producing the expected deformation, but fails to produce ARAP deformation on other meshes, either because it gives the following error:
- RuntimeError: [Open3D Error] (std::shared_ptr open3d::geometry::TriangleMesh::DeformAsRigidAsPossible(const std::vector&, const std::vector >&, size_t, open3d::geometry::MeshBase::DeformAsRigidAsPossibleEnergy, double) const) /home/runner/work/Open3D/Open3D/cpp/open3d/geometry/TriangleMeshDeformation.cpp:101: [DeformAsRigidAsPossible] Failed to build solver (factorize)
or because the single deformation produced is the translation of the handle vertex to the desired position.
I wrote the code below to illustrate the issue by following the original [example](http://www.open3d.org/docs/latest/tutorial/Advanced/mesh_deformation.html):
```
import open3d as o3d
import numpy as np
import copy
# Load meshes
myMesh = o3d.io.read_triangle_mesh("rotated_template.ply")
armadillo_mesh = o3d.io.read_triangle_mesh("Armadillo.ply")
print("myMesh mesh info: ", myMesh)
print("armadillo mesh info: ", armadillo_mesh)
# Paint meshes yellow
yellow = [0.8, 0.8, 0.1]
myMesh.paint_uniform_color(yellow)
armadillo_mesh.paint_uniform_color(yellow)
# Scale and translate myMesh mesh
myMesh.scale(45, center=myMesh.get_center())
myMesh.translate((100.0, 0.0, 0.0))
# Rotate armadillo mesh 180° about y axis
R = armadillo_mesh.get_rotation_matrix_from_xyz((0,np.pi,0))
armadillo_mesh.rotate(R)
# Create a sphere to be used as a marker
rad = 1
marker = o3d.geometry.TriangleMesh.create_sphere(radius=rad, resolution=20)
myMesh_handle_idx = 23
armadillo_handle_idx = 1593
myMesh_start_marker_pos = np.asarray(armadillo_mesh.vertices[armadillo_handle_idx])
myMesh_start_marker = copy.deepcopy(marker)
myMesh_start_marker.paint_uniform_color([0.0, 1.0, 0.0])
myMesh_start_marker.translate(myMesh_start_marker_pos)
armadillo_start_marker_pos = np.asarray(myMesh.vertices[myMesh_handle_idx])
armadillo_start_marker = copy.deepcopy(marker)
armadillo_start_marker.paint_uniform_color([0.0, 1.0, 0.0])
armadillo_start_marker.translate(armadillo_start_marker_pos)
offset = (4.0, 4.0, 4.0)
myMesh_stop_marker_pos = myMesh_start_marker_pos + offset
myMesh_stop_marker = copy.deepcopy(marker)
myMesh_stop_marker.paint_uniform_color([1.0, 0.0, 0.0])
myMesh_stop_marker.translate(myMesh_stop_marker_pos)
armadillo_stop_marker_pos = armadillo_start_marker_pos + offset
armadillo_stop_marker = copy.deepcopy(marker)
armadillo_stop_marker.paint_uniform_color([1.0, 0.0, 0.0])
armadillo_stop_marker.translate(armadillo_stop_marker_pos)
# Display meshes
o3d.visualization.draw_geometries([myMesh,
armadillo_mesh,
myMesh_start_marker,
armadillo_start_marker,
myMesh_stop_marker,
armadillo_stop_marker], mesh_show_wireframe=True)
# Perform ARAP deformation on armadillo
armadillo_vertices = np.asarray(armadillo_mesh.vertices) # Get armadillo vertices
static_ids = [idx for idx in np.where(armadillo_vertices[:, 2] < 10)[0]] # Get indexes for vertices with z < 10
static_pos = []
for id in static_ids:
static_pos.append(armadillo_vertices[id]) # Append vertex current position
constraint_ids = o3d.utility.IntVector(static_ids + [armadillo_handle_idx])
# Set the desired position for handle vertex after ARAP transformation
armadillo_handle_pos = armadillo_vertices[armadillo_handle_idx] + offset
constraint_pos = o3d.utility.Vector3dVector(static_pos + [armadillo_handle_pos])
print("Performing ARAP on Armadillo")
with o3d.utility.VerbosityContextManager(o3d.utility.VerbosityLevel.Debug) as cm:
deformed_armadillo = armadillo_mesh.deform_as_rigid_as_possible(constraint_ids,
constraint_pos,
max_iter=30)
# Perform ARAP deformation on myMesh
myMesh_vertices = np.asarray(myMesh.vertices) # Get myMesh vertices
static_ids = [idx for idx in np.where(myMesh_vertices[:, 2] < 10)[0]] # Get indexes for vertices with z < 10
static_pos = []
for id in static_ids:
static_pos.append(myMesh_vertices[id]) # Append vertex current position
constraint_ids = o3d.utility.IntVector(static_ids + [myMesh_handle_idx])
# Set the desired position for handle vertex after ARAP transformation
myMesh_handle_pos = myMesh_vertices[myMesh_handle_idx] + offset
constraint_pos = o3d.utility.Vector3dVector(static_pos + [myMesh_handle_pos])
print("Performing ARAP on MyMesh")
with o3d.utility.VerbosityContextManager(o3d.utility.VerbosityLevel.Debug) as cm:
deformed_myMesh = myMesh.deform_as_rigid_as_possible(constraint_ids,
constraint_pos,
max_iter=30)
o3d.visualization.draw_geometries([myMesh,
armadillo_mesh,
myMesh_start_marker,
armadillo_start_marker,
myMesh_stop_marker,
armadillo_stop_marker,
deformed_armadillo,
deformed_myMesh], mesh_show_wireframe=True, mesh_show_back_face=True)
```
**Console output**
```
myMesh mesh info: TriangleMesh with 75000 points and 25000 triangles.
armadillo mesh info: TriangleMesh with 172974 points and 345944 triangles.
Performing ARAP on Armadillo
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up S'
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up S'
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up system matrix L
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up system matrix L
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up sparse solver
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up sparse solver
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=0, energy=5.124176e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=1, energy=2.717055e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=2, energy=2.259923e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=3, energy=2.056035e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=4, energy=1.938971e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=5, energy=1.862051e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=6, energy=1.807044e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=7, energy=1.765457e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=8, energy=1.732802e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=9, energy=1.706462e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=10, energy=1.684791e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=11, energy=1.666682e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=12, energy=1.651359e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=13, energy=1.638256e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=14, energy=1.626951e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=15, energy=1.617117e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=16, energy=1.608503e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=17, energy=1.600907e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=18, energy=1.594169e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=19, energy=1.588161e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=20, energy=1.582775e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=21, energy=1.577924e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=22, energy=1.573538e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=23, energy=1.569555e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=24, energy=1.565924e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=25, energy=1.562604e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=26, energy=1.559557e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=27, energy=1.556753e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=28, energy=1.554165e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=29, energy=1.551770e+01
Performing ARAP on MyMesh
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up S'
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up S'
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up system matrix L
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up system matrix L
[Open3D DEBUG] [DeformAsRigidAsPossible] setting up sparse solver
[Open3D DEBUG] [DeformAsRigidAsPossible] done setting up sparse solver
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=0, energy=7.978340e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=1, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=2, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=3, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=4, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=5, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=6, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=7, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=8, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=9, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=10, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=11, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=12, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=13, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=14, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=15, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=16, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=17, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=18, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=19, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=20, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=21, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=22, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=23, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=24, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=25, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=26, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=27, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=28, energy=7.108065e+01
[Open3D DEBUG] [DeformAsRigidAsPossible] iter=29, energy=7.108065e+01
Process finished with exit code 0
```
**Images**
These are the loaded meshes
myMesh mesh info: TriangleMesh with 75000 points and 25000 triangles.
armadillo mesh info: TriangleMesh with 172974 points and 345944 triangles.

The green spheres mark the vertex selected as handle point for the transformation, the red spheres mark the desired position after the transformation


The transformation results are shown below, The yellow meshes are the original meshes, the transformation result is shown in grey



**Environment:**
- Operating system: (Ubuntu 20.04)
- Python version: (Python 3.8)
- Open3D version: 0.13.0
- open3d-python version: 0.3.0.0
- Is this remote workstation?: no
- How did you install Open3D?: (build from source)
- Compiler version (if built from source): (gcc 9.3.0)
**Additional context**
I tried already changing handle vertex, using different scales, loading different meshes with different densities but i keep getting either a "Failed to build solver" error or an incorrect deformation
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.