[EPIC] A collection of items to improve DataFuson stability (reduce effort required to upgrade)

### Is your feature request related to a problem or challenge?

This is broken out from a more general ticket here
- https://github.com/apache/datafusion/issues/13525

🥳 In my opinion DataFusion is now good enough (performance and feature wise) for many people to have buit real systems and products

However, as more people build "real" systems using DataFusion, our historic "move fast and break things and hope you can keep up" mentality likely needs to adjust to a more mature "move as fast as possible, but minimize breakages" type response.

My summary of the discussion on https://github.com/apache/datafusion/issues/13525 from @findepi @scsmithr @waynexia @timsaucer @Rachelint @Omega359 @jonmmease @Dandandan and @andygrove was that many existing heavy users of DataFusion spend a lot of time during upgrades from one DataFusion release to another

Specifically, I think the core challenge I heard was *NOT* the mechnical API changes required, but the effort required to diagnose more suble issues such as:
- Queries that used to work stopped working

### Describe the solution you'd like

I would like to improve the ease of upgrading DataFusion versions

There are many ways to do so and I would like to use this ticket to capture / organize the work in this area

### Related Items
- [ ] https://github.com/apache/datafusion/issues/13525
- [x] https://github.com/apache/datafusion/issues/13504
- [x] https://github.com/apache/datafusion/issues/13625
- [x] https://github.com/apache/datafusion/issues/12357
- [x] https://github.com/apache/datafusion/issues/5269
- [x] https://github.com/apache/datafusion/issues/13652
- [x] https://github.com/apache/datafusion/issues/13811
- [ ] https://github.com/apache/datafusion/issues/13661
- [x] https://github.com/apache/datafusion/issues/13665
- [x] https://github.com/apache/datafusion/pull/13701
- [ ] https://github.com/apache/datafusion/issues/16622
- [ ] https://github.com/apache/datafusion/pull/16541

### Additional testing
- [x] https://github.com/apache/datafusion/issues/13470
- [ ] https://github.com/apache/datafusion/issues/12115
- [x] https://github.com/apache/datafusion/issues/12114

### More Context:

Start by reading the discussion in issue 13525 and the related issues and pull requests listed in this epic. Determine which upgrade-stability work remains open, then define a focused, independently actionable item with clear completion criteria rather than working on the epic as a whole.

Written by the indexing model from the issue text.