PointCloudLibrary / PointCloudLibrary/pcl

pcl viewer, plyWriter, and external viewer clouds are inconsistent

Open
#1,474 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind: bug module: io needs: testing
Dominant language
C++
Stars
11.1k
Forks
4.7k
Avg merge
4d 10h
Merged PRs (30d)
6

Description

I have a point cloud which is reading data from a kinect and it appears perfect in the pcl visualizer and viewer. However, when I write it out with plyWriter and open it in an external viewer, such as meshlab, the depth is inverted and the faces/normals are the reverse of what they should be.

Are the inverted normals and depth expected behavior with a known solution or an inconsistency/bug between how clouds are rendered and written to ply files? Also, how can I work around this?

I've tried varying the orientation quaternion entries with wxyz values of [1 0 0 0], [0 1 0 0], [0 0 1 0], respectively in the call of WritePLY::write, but this did not appear to have any effect.

It may be worth noting that I'm using libfreenect2 and libfreenect2pclgrabber to capture the images and transfer the data to pcl before writing the ply.

Here are a few images shortly that illustrate the problem clearly:

Setup

Sorry, didn't get the cereal box that appears below in my setup photo, please forgive the oversight. The general idea should be clear, however.
tiki_scene_close_setup

Front

Note how the pclviewer image is correct, but the meshlab image is inverted, when compared to the in memory view produced by pcl. In the meshlab image the front is brighter (i.e. the front/normal direction). The data is not modified before the rendering and call to writeply.

Pclviewer:

kinect_feed00

Meshlab:

inverted_mesh00

Back

Note how the pcl viewer appears correct, while the front of the ply data is inverted. This can also be seen readily in the side normals view below. Also note the color brightness difference in meshlab, the brighter side, on the inside of the wall, box, etc, represents the front of the mesh faces and the darker side is the back.

pcl viewer:
kinect_feed01

Meshlab:
inverted_mesh01

Side View with Normals

Here is the meshlab side view with normals. Note how the meshlab normals are pointing into the wall rather than out of the wall, which is the expected behavior. I'm working on generating equivalent pcl normals but don't have that ready yet.

Meshlab:

inverted_mesh03

Thanks for your time and help!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with io/include/pcl/io/ply_io.h, especially WritePLY::write, and compare its output with the rendering path in visualization/include/pcl/visualization/pcl_visualizer.h. Reproduce the Kinect cloud workflow and inspect how orientation, depth, faces, and normals differ between PCL and MeshLab. Done means the inconsistency is identified and fixed or its expected behavior and workaround are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.