Split up scan node in Polars DSL
Open
cudf-polars
improvement
Python
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
The differing logic for the various file types in the scan node is enough to make that function very difficult to parse, and it is made doubly problematic by the issues it causes for reusing variable names for type checkers.
All of these options naming changes suggest we should split this function out of a single massive monolithic if/else for the different IO types.
_Originally posted by @wence- in https://github.com/rapidsai/cudf/pull/20224#discussion_r2419226452_
Contributor guide
Assessment
This issue has not been assessed yet.