NVIDIA / NVIDIA/cudf

[FEA] Implement merged 'mega' kernel to parse leaf-level columns in JSON reader

Open
#16,965 0 comments 0 reactions 1 assignee Claimed by @shrshi View on GitHub
cuIO feature request libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
Inferring types and parsing leaf level columns in the JSON reader launches separate kernels for each column.
![Image](https://github.com/user-attachments/assets/68458944-77ad-4aab-84ba-91c75421b48f)

We can obtain improved performance by gathering the offsets for columns contiguously, and then parsing them in a single kernel.

**Describe the solution you'd like**
Partitioning strategies to consider:
- For parsing, 1 thread per offset.
- 1 warp / column (but 32 offsets/warp), consecutive warps will most likely access nearby memory and probably benefit from coalescing
- Fixed number of characters per thread, but more careful thought is required for distributing work depending on column type.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.