apache / apache/datafusion-ballista

Add suport for `DataFrame.checkpoint()`

Open
#1,937 0 comments 0 reactions 1 assignee Claimed by @sandugood View on GitHub
enhancement
Dominant language
Rust
Stars
2.1k
Forks
320
Avg merge
1d 22h
Merged PRs (30d)
66

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Add support for checkpointing the DataFrame - so essentially writing the intermediate representation of a plan to disk and breaking lineage.

**Describe the solution you'd like**
Introducing a `DataFrame.checkpoint()` method

**Describe alternatives you've considered**

**Additional context**
Related to #1829, but split into two parts.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.