npz support
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 5.8k
- Forks
- 678
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 27
Description
npz format is basically a zipped npy. I've seen other users who are interested in this as well, so I hope you consider this as a long term enhancement request.
Why would this be useful? Using compressed numpy can save a lot of bandwidth on the network and do the de-compression on the gpu. Also, numpy is pretty standard.
PS. Its also worth noting another nvidia project: nvcomp that might help in enabling this similar to how you are using nvjpeg.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is identified. Start by locating DALI's existing NPY handling and reviewing the referenced nvjpeg and nvcomp approaches; the scope, integration point, and acceptance criteria for NPZ support still need to be defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100