PointCloudLibrary / PointCloudLibrary/pcl

[SupervoxelClustering] setIndices not working on SupervoxelClustering Classs

Open
#4,686 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind: bug module: segmentation
Dominant language
C++
Stars
11.1k
Forks
4.7k
Avg merge
4d 10h
Merged PRs (30d)
6

Description

Describe the bug
When using the superVoxelClustering class to segment objects in a scene using LCCP, I'd like to do so only on a subset of the points, (ie, removing the indices that correspond to the plane, and/or those too far away from the camera). I obtain these desired indices to be considered and pass them to the class using the setIndices function, but the supervoxels get computed for the totality of the input pointcloud, and therefore also the LCCP segmentation. I need the output to be in form of indices from the original pointcloud.

Context
My goal is to perform LCCP segmentation on a subset of indices of the original pointcloud.

Expected behavior
Like most other PCL functions, superVoxelClustering inherits setIndices from PCLBase. However, it seems to ignore this indices, since the result is computed on the whole input pointcloud

Current Behavior
Instead of computing the superVoxelClustering using the subset of indices passed via setIndices, it computes it for all the pointcloud.

To Reproduce
Same code as the lccp example,
https://github.com/PointCloudLibrary/pcl/blob/master/examples/segmentation/example_lccp_segmentation.cpp
where lines 272 to 319 have been modified as follows, to make use of indices:

(input pointcloud is any pointcloud with a plane and some objects on top of it. )

// Find plane  in pointcloud
pcl::PointIndices::Ptr plane_ids(new pcl::PointIndices);
pcl::ModelCoefficients::Ptr coeffs(new pcl::ModelCoefficients);
pcl::SACSegmentation<pcl::PointXYZRGBNormal> seg;
seg.setOptimizeCoefficients(true);
seg.setMaxIterations(200);
seg.setModelType(pcl::SACMODEL_PLANE);
seg.setMethodType(pcl::SAC_RANSAC);
seg.setDistanceThreshold(0.01);
seg.setInputCloud(input_pointcloud);
seg.segment(*plane_ids, *coeffs);

// Extract indices of points not in plane
pcl::PointIndices::Ptr non_plane_ids(new pcl::PointIndices);
pcl::PointCloud<pcl::PointXYZRGBNormal>::Ptr non_plane_cloud(new pcl::PointCloud<pcl::PointXYZRGBNormal>);
pcl::ExtractIndices<pcl::PointXYZRGBNormal> extract;
extract.setInputCloud(input_pointcloud);
extract.setIndices(plane_ids);
extract.setNegative(true);
extract.filter(non_plane_ids->indices);
extract.filter(*non_cloud_plane);

// Split pointcloud to fit SuperVoxelClustering input format
pcl::PointCloud<PointT>::Ptr cloud_RGB(new pcl::PointCloud<PointT>);
pcl::PointCloud<pcl::Normal>::Ptr cloud_normals(new pcl::PointCloud<pcl::Normal>);
pcl::copyPointCloud(*input_pointcloud, *cloud_RGB);
pcl::copyPointCloud(*input_pointcloud, *cloud_normals);

// Compute SuperVoxel Clustering
pcl::SupervoxelClustering<PointT> super(0.005, 0.02);
super.setUseSingleCameraTransform(false);
super.setInputCloud(cloud_RGB);
super.setNormalCloud(cloud_normals);
super.setIndices(non_plane_ids);         // <------------------- XXXX This is the line where the bug is XXXXX
super.setColorImportance(0.0f);
super.setSpatialImportance(1.0f);
super.setNormalImportance(4.0f);
std::map<std::uint32_t, pcl::Supervoxel<PointT>::Ptr> supervoxel_clusters;
PCL_INFO("Extracting supervoxels\n");
super.extract(supervoxel_clusters);
super.refineSupervoxels(2, supervoxel_clusters);

PCL_INFO("Getting supervoxel adjacency\n");
std::multimap<std::uint32_t, std::uint32_t> supervoxel_adjacency;
super.getSupervoxelAdjacency(supervoxel_adjacency);

PCL_INFO("Starting LCCP Segmentation\n");
pcl::LCCPSegmentation<PointT> lccp;
lccp.setConcavityToleranceThreshold(10.0);
lccp.setSanityCheck(false);
lccp.setSmoothnessCheck(true, 0.005, 0.02, 0.1);
lccp.setKFactor(1);
lccp.setInputSupervoxels(supervoxel_clusters, supervoxel_adjacency);
lccp.setMinSegmentSize(5);
lccp.segment();

PCL_INFO("Interpolation voxel cloud -> input cloud and relabeling\n");
pcl::PointCloud<pcl::PointXYZL>::Ptr sv_labeled_cloud = super.getLabeledCloud();
pcl::PointCloud<pcl::PointXYZL>::Ptr lccp_labeled_cloud = sv_labeled_cloud->makeShared();
lccp.relabelCloud(*lccp_labeled_cloud);
SuperVoxelAdjacencyList sv_adjacency_list;
lccp.getSVAdjacencyList (sv_adjacency_list);  // Needed for visualization

Screenshots/Code snippets
Example (but repeats in every example):
Original Pointcloud (input_pointcloud):
oringal_pointcloud

Pointcloud after plane removal (non_plane_cloud)
pointcloud_no_plane

Supervoxel clustering results (despite setIndices(non_plane_ids):
supervoxel_results

LCCP Segmentation results:
lccp_results

Your Environment (please complete the following information):

  • OS: Ubuntu 18.04
  • Compiler:GCC 7.5
  • PCL Version 1.8

Possible Solution

Not obligatory, but suggest a fix/reason for the bug. Feel free to create a PR if you feel comfortable.

Additional context

Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/segmentation/example_lccp_segmentation.cpp and trace SupervoxelClustering::setIndices through extract(), using the issue's modified code to reproduce the behavior. Done means clustering and subsequent LCCP segmentation use only the supplied subset while preserving indices from the original point cloud.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
computer-vision
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.