PointCloudLibrary / PointCloudLibrary/pcl

PCDWriter::writeBinaryCompressed cannot write large pcd file due to 32-bit limitation

Open
#2,152 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs: code review
Dominant language
C++
Stars
11.1k
Forks
4.7k
Avg merge
4d 10h
Merged PRs (30d)
6

Description

PCDWriter::writeBinaryCompressed cannot write large pcd file due to 32-bit size limitation:

(gdb) l 380
375	  }
376	
377	  char* temp_buf = static_cast<char*> (malloc (static_cast<size_t> (static_cast<float> (data_size) * 1.5f + 8.0f)));
378	  // Compress the valid data
379	  unsigned int compressed_size = pcl::lzfCompress (only_valid_data, 
380	                                                   static_cast<uint32_t> (data_size), 
381	                                                   &temp_buf[8], 
382	                                                   static_cast<uint32_t> (static_cast<float>(data_size) * 1.5f));
383	  unsigned int compressed_final_size = 0;
384	  // Was the compression successful?
(gdb) p data_size
$13 = 2935264192
(gdb) p static_cast<uint32_t> (static_cast<float>(data_size) * 1.5f)
$14 = 107928576

(note that 2935264192*1.5=4402896288, which overflowed on an uint32_t value)

... and this will generate an error message like:
[pcl::lzf_compress] Attempting to write data outside the output buffer!
terminate called after throwing an instance of 'pcl::IOException'
what(): : [pcl::PCDWriter::writeBinaryCompressed] Error during compression!

Your Environment

  • Operating System and version: Ubuntu 14.04
  • Compiler: gcc 4.8.4
  • PCL Version: 1.8.1

Expected Behavior

Write the large .pcd file correctly.

Current Behavior

Generate the following exception and crash:
[pcl::lzf_compress] Attempting to write data outside the output buffer!
terminate called after throwing an instance of 'pcl::IOException'
what(): : [pcl::PCDWriter::writeBinaryCompressed] Error during compression!

Possible Solution

Replace the 32-bit size value used by lzfCompress() to 64-bit (size_t or uint64_t).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at PCDWriter::writeBinaryCompressed and read the pcl::lzfCompress interface and its size handling. Trace data_size, temporary-buffer allocation, and compressed-size calculations for the reported 32-bit overflow. Done means large PCD files compress and write successfully without truncation or the reported compression exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.