Add `DataFrame::map` utility .map function for DataFrame for modifying internal LogicalPlan
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Currently when applying custom `LogicalPlan`s (extensions or transformation functions) on `DataFrame`s I need to convert `DataFrame` to a `LogicalPlan` and then back to a `DataFrame` making cumbersome code.
### Describe the solution you'd like
It would be very helpful to have a `.map` function which transforms the `LogicalPlan` inside `DataFrame`s without reconstructing it.
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start by locating the DataFrame and LogicalPlan entry points in the DataFusion codebase and read how conversion between them currently works. Determine the public map API and its expected behavior for transforming an internal LogicalPlan; done means callers can apply custom LogicalPlan transformations without manually converting to and from DataFrame.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design, data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100