Restructure core codepaths to prevent stack overflows

### Describe the bug

In PR [#16506](https://github.com/apache/datafusion/pull/16506), we added `#[recursive]` annotations (behind the `recursive_protection` feature flag) to a subset of recursive functions across the optimizer, physical planner, and expression evaluator. While this prevented stack-overflow panics on deeply nested plans, benchmarks showed a 1–2% slowdown in planning time when the feature is enabled.

> “The benchmarks seem to imply this has some non trivial overhead (slows down planning by 1–2%).
> I wonder if there is any way to restructure the affected codepaths to avoid the recursion (change them into an iterative version, for example, or Box more structures to reduce the stack size?)”
> — @alamb
>

This epic will serve as a central place to explore multiple approaches (e.g., iterative rewrites, Box-based data structures to shrink stack usage, or other patterns) to reduce/eliminate deep recursion in key codepaths, remove the need for defensive macros, and regain any lost performance, all while still handling arbitrarily nested plans reliably.

### To Reproduce

https://github.com/apache/datafusion/pull/16787 provides a reproducer.

### Expected behavior

DataFusion doesn't overflow its stack.

### Additional context

See the discussion in [#16506](https://github.com/apache/datafusion/pull/16506)

Start with the reproducer in PR #16787 and read the recursion changes in PRs #16506 and #16787. Trace the affected optimizer, physical planner, and expression evaluator codepaths, including the recursive_protection feature, then compare iterative, boxed, or other approaches. Done means deeply nested plans avoid stack overflows without the defensive macros or the reported planning slowdown.

Written by the indexing model from the issue text.