NVIDIA / NVIDIA/cudf

[FEA] Add Avro reader benchmarks to the cuIO benchmarking suite

Open
#14,574 1 comment 0 reactions 0 assignees View on GitHub
0 - Backlog cuIO feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
We have reader benchmarks for CSV, JSON, Parquet and ORC in the cuIO nvbench benchmarking suite. We should add benchmarking for the Avro reader.

The cuIO benchmarks are located here:
https://github.com/rapidsai/cudf/tree/branch-24.02/cpp/benchmarks/io

Unfortunately, we don't have an Avro writer implementation in libcudf, so the naive approach of modeling benchmarks after [json_reader_input.cpp](https://github.com/rapidsai/cudf/blob/branch-24.02/cpp/benchmarks/io/json/json_reader_input.cpp) will not work.

**Describe the solution you'd like**
Our options would be:
* add an MVP Avro writer to libcudf
* add/use a dependency to write Avro files
* maintain a repository of large Avro files (>100 MB) for benchmarking purposes

**Describe alternatives you've considered**
Continue without automated benchmarks for the Avro reader

**Additional context**
The libcudf Avro reader does not support nested types so the benchmarks should start by only covering primitive types.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.