apache / apache/datafusion-ballista
Add suport for `DataFrame.checkpoint()`
Open
enhancement
- Dominant language
- Rust
- Stars
- 2.1k
- Forks
- 320
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 66
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Add support for checkpointing the DataFrame - so essentially writing the intermediate representation of a plan to disk and breaking lineage.
**Describe the solution you'd like**
Introducing a `DataFrame.checkpoint()` method
**Describe alternatives you've considered**
**Additional context**
Related to #1829, but split into two parts.
Contributor guide
Assessment
This issue has not been assessed yet.