NVIDIA / NVIDIA/CUDALibrarySamples
[FEA] Requesting "fall back" CPU implementations of certain decompression algorithms
@eschmidt-nvidia is already working on this.
Since Mar 13, 2023.
- Dominant language
- Cuda
- Stars
- 2.5k
- Forks
- 478
- PR merge metrics
- No merged PRs in 30d
Description
Is your feature request related to a problem? Please describe.
No problems...
Describe the solution you'd like
It would be very beneficial to have CPU-only implementations of nvCOMP decompression algorithms. My limited understanding is that some compressed streams created my nvCOMP are not a 'compliant' byte stream, meaning for example a standard CPU-based lz4 decompressor may not be able to handle nvCOMP compressed streams due to additional information embedded in the stream (or something along those lines).
Additional context
NCAR recently became aware of the power of nvCOMP in an Open Hackathons event. Coupled with GPU-Direct Storage (GDS), this could be a very powerful tool for the Earth Sciences community. In order to deploy nvCOMP in this setting, however, it is imperative that any compressed files created through nvCOMP be usable on any machine, in particular machines with no GPUs whatsoever. Thus, an enabling capability for this use would be the ability for nvCOMP to decompress its own streams, on systems without GPUs.
Describe alternatives you've considered
It could be sufficient for nvCOMP to call pre-existing CPU decompressors, after it has handled any implementation-specific stream issues.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.