NVIDIA / NVIDIA/CUDALibrarySamples

Maximum data size for compression

Open
#326 5 comments 0 reactions 1 assignee View on GitHub

@naveenaero is already working on this.

Since Apr 29, 2026.

nvCOMP question
Dominant language
Cuda
Stars
2.5k
Forks
478
PR merge metrics
No merged PRs in 30d

Description

I have integrated nvCOMP (version nvcomp_3.0.5_windows_12.x) into our system for the compression of scientific data (GIS, simulation, …) and have noticed that compression fails for data of 2^32 bytes or more, while the compression of a buffer of size 2^32 - 1 works flawlessly.

Specifically, nvcompManagerBase::compress throws the exception "Encountered Cuda Error: 2: 'out of memory'." after calling nvcompManagerBase::configure_compression with a decomp_buffer_size >= 2^32.

If the chosen compression manager is a DeflateManager or GdeflateManager, the call of compress first outputs a warning to std::cerr:

"WARNING: In nvcompBatchedGdeflateCompressGetTempSize: Might not have enough memory available on this GPU.
Require 22.002 GB but detected only 0 GB available"

Both of the reported numbers in this case seem wrong to me, but this is not my main issue, as we are not planning to use Deflate/Gdeflate anyways.

Is this an undocumented hard limit for the file size that nvCOMP can handle? I don't mind if there is a limit of 4 GB to the buffer size even with much more available memory. However, in this case, it would be nice to know and query this limit beforehand so to not run into an exception.

This has been tested on a system with:

0 - Device name: NVIDIA GeForce RTX 3060
0 - CUDA Driver Version / Runtime Version: 12.3 / 12.3
0 - CUDA Capability Major/Minor version number: 8.6
0 - Total amount of global memory: 12.00 GBytes

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.