NVIDIA / NVIDIA/CUDALibrarySamples

[BUG] Maximum allowed chunk size for gdeflate is 64KB

Open
#327 6 comments 0 reactions 1 assignee View on GitHub

@naveenaero is already working on this.

Since Apr 29, 2026.

bug nvCOMP
Dominant language
Cuda
Stars
2.5k
Forks
478
PR merge metrics
No merged PRs in 30d

Description

The blog post states

The following examples use the high throughput GDeflate compression format. [...] Larger chunk sizes typically lead to higher compression ratios at the expense of less parallelism exposed to the GPU. A good starting chunk size is 64 KB, but feel free to experiment with these values to explore the associated tradeoffs for your datasets.

In my case I'm using e.g. 40 managers to process >11GB, so I already have enough parallelism and am trying to get better CR than 0.92

Similarly, the changelog for 2.4.1 states

The Deflate batched decompression API can work on uncomprressed data chunk larger than 64KB.

However, actually using 256KB (GdeflateManager nvcomp_manager{ 1 << 18, nvcompBatchedGdeflateDefaultOpts, stream, NoComputeNoVerify };) leads to this error:

[7] [2024-01-03 20:49] [critical] nvCOMP version 3.0.0; Linux x86-64; CUDA 11.8 build
[7] [2024-01-03 20:49] [error] In nvcompBatchedGdeflateCompressGetOutputSize(): Maximum allowed chunk size for gdeflate is 64KB
[7] [2024-01-03 20:49] [error] In nvcompBatchedGdeflateCompressGetTempSize(): Maximum allowed chunk size for gdeflate is 64KB
[7] [2024-01-03 20:49] [error] In nvcompBatchedGdeflateCompressAsync(): Maximum allowed chunk size for gdeflate is 64KB

Is >64KB only supported for deflate, not gdeflate?

Steps/Code to reproduce bug
GdeflateManager nvcomp_manager{ 1 << 18, nvcompBatchedGdeflateDefaultOpts, stream, NoComputeNoVerify };

Expected behavior
256KB uncomp_chunk_size should work.

Environment details (please complete the following information):

  • Environment location: Docker in GCP VM
  • Method of nvCOMP install: without extensions

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.