GPU Compressors
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 80
- Forks
- 8
- Avg merge
- 21h 44m
- Merged PRs (30d)
- 5
Description
See https://github.com/NVIDIA/CUDALibrarySamples/tree/master/nvCOMP and https://github.com/rapidsai/kvikio/tree/branch-25.02/python/kvikio and https://github.com/NVIDIA/nvcomp?tab=readme-ov-file
Python bindings already exist so I think we would need rust bindings as well (although maybe we could call the python through rust with pyo3? Maybe a bad performance?). In any case, it would be great to somehow add support for this (somehow emphasized, almost certainly a lot of work) given the speed boost of not having to do the CPU bounce, and zarr-python v3 natively supprting GPU buffers if I'm not mistaken. So this should be doable, but probably a lot of work and maybe it's own package
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the linked NVIDIA nvCOMP samples, kvikio Python bindings, and nvCOMP documentation, then review how zarr-python v3 handles GPU buffers. Determine whether Rust bindings or a separate package is needed and define done as GPU compression support without a CPU bounce; the issue does not specify files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100