NVIDIA / NVIDIA/cudf

[BUG] cuIO benchmarks generate larger files than expected

Open
#11,204 1 comment 0 reactions 0 assignees View on GitHub
bug cuIO libcudf tests
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

cuIO ORC and Parquet benchmarks generate larger files than previously (e.g. GDS blog data).
Some observations:

- [x] Only the cases where both cardinality and run length are set lead to small files.
- [ ] Cases where the data does not encode/compress well now generate files larger by ~20% (based on integral columns).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.