Collecting ideas to improve the PCD file format

Currently, the PCD file format is at version 0.7 (and has been for many years.
This issue represents a collection of ideas for improving the file format. Everyone is welcome to suggest ideas or comment on ideas. It would also be helpful if people comment which ideas are most important in their opinion.
These ideas (or some of them) could some day be implemented in a version 0.8 of the PCD file format.

• binary compressed data: currently, this uses two 32-bit unsigned binary numbers to specify how large the compressed and uncompressed data is. This should instead be two 64-bit unsigned binary numbers, to allow very large datasets. It might make sense to add a new binary_compressed64 mode for this. Additionally, it could be advantageous if the data can be stored in several sections, where the data in each section is spatially close and can be decompressed independently. Then the reader does not have to decompress all the data at once, which might not fit into memory (in case of very large clouds) (related: https://github.com/PointCloudLibrary/pcl/issues/2152)
• additional header entries: currently, there is no possibility to store sequence number, timestamp, and frame id (related: https://github.com/PointCloudLibrary/pcl/issues/6152)
• it could make sense to make some header entries optional, meaning that it is allowed that the entry does not appear, in which case a certain default value is assumed
• officially declare lines starting with # as comments?
• maybe make 0.8 a superset of 0.7, so that every valid PCD 0.7 file is also a valid PCD 0.8 file?
• currently, binary_compressed uses the LZF algorithm, however, implementations of this may not be available in many programming languages (other than C/C++). Consider adding a different compressed format (could be binary_compressed64) which uses a more widely available compression algorithm

It would be good if we can create a list of software that can read or write PCD files, so that we can notify them in case of a PCD version 0.8:

• https://github.com/PDAL/PDAL
• https://github.com/CloudCompare/CloudCompare

Start by reviewing the linked PCD file format specification and related issues #2152 and #6152, then inventory the listed compatibility and format proposals. This issue is done only when the project agrees on a concrete, scoped PCD 0.8 design and identifies affected readers and writers.