NVIDIA / NVIDIA/cudf

[FEA] Should construction of `cudf.DataFrame` and et. al. public objects just use pandas constructors internally?

Open
#19,894 0 comments 0 reactions 0 assignees View on GitHub
cudf.pandas feature request Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
Currently, `DataFrame.__init__` and friends largely re-implement similar logic to their pandas counterparts accounting for all the constructor arguments. While we have tests verifying similarity for a lot of constructor behaviors between cuDF and pandas:

1. There's probably a long tail of constructor behaviors we probably haven't matched, accounting for all the permutations of keyword arguments.
2. All constructors eventually call `as_column` to parse "un-typed" (e.g. Python lists of Python objects) data which probably doesn't perfectly match pandas' parsing of "un-typed" data

**Describe the solution you'd like**
Constructors implemented as e.g.

```python
class DataFrame(...):

def __init__(self, data, index, columns, copy):
pd_object = pd.DataFrame(data, index, columns, copy)
# logic to convert pd_object into a representable cuDF object
```

Some caveats:
1. Currently cuDF objects accept a wider range of objects compared to pandas, namely `pyarrow` objects and objects implementing `__cuda_array_interface__`. _If_ `cudf.pandas` is meant to be the future interface of cuDF classic, then it would be reasonable to eventually disallow these object in the constructor to strictly match pandas. Otherwise, we'd need to probably maintain the current constructor logic.
2. When using `cudf.pandas`, we'd need to account for arguments being proxy pandas objects. If we used the proposed construction pattern for correctness with pandas, going through pandas would probably invoke a DTHTD roundtrip for our proxy objects

**Describe alternatives you've considered**
Status quo

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.