Logical Expr serde should be stack safe
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
Noticed while working on https://github.com/apache/arrow-datafusion/issues/5691 I was getting stack overflows while running tests (well actually a segfault but after some investigation I think it was just a quirk of my ARM mac and really the underlying issue was a stack overflow). I was able to run the tests by compiling them in release mode but the current recursive expression parsing/translation is not stack safe and will likely eventually cause issues.
Otherwise deeply nested expressions can cause a system crash.
### Describe the solution you'd like
Avoid recursion in `datafusion_proto::logical_plan::from_proto::parse_expr`
### Describe alternatives you've considered
Leave it as is and document as a known limitation
### Additional context
_No response_
Contributor guide
Research direction
Start at datafusion_proto::logical_plan::from_proto::parse_expr and inspect the recursive expression parsing and translation described in the issue. Reproduce the deeply nested expression failure using the relevant tests, then verify that parsing such expressions no longer overflows the stack.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100