NVIDIA / NVIDIA/cudf

[FEA] Add `PartitionedSink` support to cudf-polars

Open
#23,004 0 comments 0 reactions 1 assignee Claimed by @Matt711 View on GitHub
cudf-polars feature request
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
Support for hive-style partitioned sinks

```python
lf.sink_parquet(pl.PartitionBy("./output/", key=["year", "month"]))
# writes: ./output/year=2024/month=01/00000001.parquet
# ./output/year=2024/month=02/00000001.parquet
# ...
```

**Describe the solution you'd like**
A new dedicated IR node: `PartitionedSink`.

**Describe alternatives you've considered**
We could use existing `Sink` but I don't like that because I think `Sink` and `PartitionedSink` will be different enough that they warrant different IR.

**Additional context**
Depends on polars changes, so will need a new pinning first PR

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.