NVIDIA / NVIDIA/cudf

Skip non-empty nulls when creating mixed type columns in `read_json`

Open
#19,899 0 comments 0 reactions 0 assignees View on GitHub
cuIO improvement libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Proposal**

Currently, while constructing device_json_column in `host_tree_algorithms.cu`, non-empty nulls are being added for list-type columns when propagating parent's null mask. This can be optimized by skipping these non-empty nulls and instead propagating the null mask directly to the list-type children before creating the columns.

**Suggested changes**
- In `cudf/cpp/src/io/json/host_tree_algorithms.cu`, particularly around line 948:
- Instead of setting the parent's null mask to null for mixed-type list rows, propagate the parent's null mask to the list type children and create columns accordingly.
- This will help skip adding non-empty nulls for list children and optimize device column construction.

**Benefits**
- Avoids unnecessary non-empty nulls in device columns
- More accurate null propagation for nested list columns
- Potential performance and memory improvements for large JSON datasets

**References**
- User suggestion from [discussion](https://github.com/rapidsai/cudf/pull/12873#discussion_r2323749495)
- Related code: `host_tree_algorithms.cu` line 948

---
*Created automatically from user feedback and code review analysis.*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.