NVIDIA / NVIDIA/cudf

Split up scan node in Polars DSL

Open
#20,229 0 comments 0 reactions 0 assignees View on GitHub
cudf-polars improvement Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

The differing logic for the various file types in the scan node is enough to make that function very difficult to parse, and it is made doubly problematic by the issues it causes for reusing variable names for type checkers.

All of these options naming changes suggest we should split this function out of a single massive monolithic if/else for the different IO types.

_Originally posted by @wence- in https://github.com/rapidsai/cudf/pull/20224#discussion_r2419226452_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.