NVIDIA / NVIDIA/CUDALibrarySamples
Maximum data size for compression
@naveenaero is already working on this.
Since Apr 29, 2026.
- Dominant language
- Cuda
- Stars
- 2.5k
- Forks
- 478
- PR merge metrics
- No merged PRs in 30d
Description
I have integrated nvCOMP (version nvcomp_3.0.5_windows_12.x) into our system for the compression of scientific data (GIS, simulation, …) and have noticed that compression fails for data of 2^32 bytes or more, while the compression of a buffer of size 2^32 - 1 works flawlessly.
Specifically, nvcompManagerBase::compress throws the exception "Encountered Cuda Error: 2: 'out of memory'." after calling nvcompManagerBase::configure_compression with a decomp_buffer_size >= 2^32.
If the chosen compression manager is a DeflateManager or GdeflateManager, the call of compress first outputs a warning to std::cerr:
"WARNING: In nvcompBatchedGdeflateCompressGetTempSize: Might not have enough memory available on this GPU.
Require 22.002 GB but detected only 0 GB available"
Both of the reported numbers in this case seem wrong to me, but this is not my main issue, as we are not planning to use Deflate/Gdeflate anyways.
Is this an undocumented hard limit for the file size that nvCOMP can handle? I don't mind if there is a limit of 4 GB to the buffer size even with much more available memory. However, in this case, it would be nice to know and query this limit beforehand so to not run into an exception.
This has been tested on a system with:
0 - Device name: NVIDIA GeForce RTX 3060
0 - CUDA Driver Version / Runtime Version: 12.3 / 12.3
0 - CUDA Capability Major/Minor version number: 8.6
0 - Total amount of global memory: 12.00 GBytes
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.