Normals at edges all go trough origin

The issue is best illustrated in an image:

Your Environment

• Operating System and version: Ubuntu 18.04
• Compiler: Used Ubuntu pre built binaries
• PCL Version: 1.8.1

Context

I try to estimate the normals of some bricks lying on the floor. But as seen in the image all the normals at edges connect to the origin.

Expected Behavior

Proper normals estimation

Current Behavior

Code to Reproduce

void compute_normals(pcl::IntegralImageNormalEstimation<pcl::PointXYZRGB, pcl::Normal>& ne, pcl::visualization::PCLVisualizer& viewer,
                     pcl::PointCloud<pcl::PointXYZRGB>::Ptr& input_cloud, pcl::PointCloud<pcl::Normal>::Ptr& output_cloud, float normal_smoothing_size,
                     int normal_cloud_visualization_densitiy){
  ros::WallTime start = ros::WallTime::now();
  ne.setNormalSmoothingSize(normal_smoothing_size);
  ne.setInputCloud(input_cloud);
  ne.compute(*output_cloud);
  viewer.removeAllPointClouds();
  viewer.addPointCloudNormals<pcl::PointXYZRGB, pcl::Normal>(input_cloud, output_cloud, normal_cloud_visualization_densitiy);
  viewer.spinOnce ();
  stop(start, "Normals");
}

I could upload a rosbag if needed.

Possible Solution

I have no clue, but I saw that the neighborhoud search radius makes the problem worse the bigger it is. The above image uses a radius of 15. The one below uses a radius of 5:

Start with the compute_normals entry point and the pcl::IntegralImageNormalEstimation call shown in the issue. Reproduce the edge-normal behavior with an equivalent point cloud and compare the neighborhood sizes of 15 and 5; done means edge normals no longer connect to the origin while normal estimation remains correct.