apache / apache/datafusion-ballista
Enable `View` types
- Dominant language
- Rust
- Stars
- 2.1k
- Forks
- 320
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 66
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
`ViewTypes` has been disabled in #1185 due to Arrow IPC `ViewType` serialisation issues (#1182).
**Describe the solution you'd like**
As pointed out by @mbutrovich in https://discord.com/channels/885562378132000778/1408110811637088286/1408118010161401958 view types can be garbage collected before arrow IPC called. His implementation can be seen at https://github.com/mbutrovich/datafusion-comet/commit/a8cbfb7a7c6071b61f3bf0ab28e5d87fdf8a5639
**Describe alternatives you've considered**
**Additional context**
- https://github.com/apache/arrow-rs/issues/7185
Contributor guide
Research direction
Start by tracing the disabled `ViewTypes` support and the Arrow IPC `ViewType` serialization path, then read Arrow issue #7185 and the linked datafusion-comet implementation. Done means view types are enabled and remain valid through Arrow IPC serialization without the reported garbage-collection failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100