[EPIC] A collection of items to improve developer / CI speed

### Is your feature request related to a problem or challenge?

As DataFusion becomes more mature and gets more features and tests, the amount of time it takes to build and test the system has been increasing

This has several downsides:
## Barrier to new contributors is higher

The resources required to build / link DataFusion are now very large which means some people may not be able to run them. For example @Alexhuszagh reports on https://github.com/apache/datafusion/pull/13693#issuecomment-2526394913

> I only tested this locally with the failing datasource::physical_plan::json::tests::test_chunked_json tests, but it should be the same with all the remaining errors. My test machine doesn't have enough RAM to link the entire core test suite,

## We are likely wasting lots of resources running CI tests
By my count the CI checks on the most recent checkin ([link](https://github.com/apache/datafusion/pull/13797/checks)) require over 200 hours of runner time

We are lucky the ASF gets many runner credits from github, but this level is very wasteful in my opinion and likely unsustainable as we contemplate adding additional testing such as
- https://github.com/apache/datafusion/issues/13812

## Larger binary size
I have noticed that the datafusion-cli binary is now almost 100MB it used to be 60MB

Also, people such as @g3blv have noticed that the WASM build has increased 50%:
https://github.com/apache/datafusion/discussions/9834#discussioncomment-11574581

See
- https://github.com/apache/datafusion/issues/13816

## Decreased developer productivity due to longer cycle times
As DataFusion gets bigger, I have noticed that recompiling datafusion for me takes longer and longer

cargo test ...

### Describe the solution you'd like

I would like to make the building and testing of DataFusion "easier" and leaner. This ticket tracks work to improve things in this area

### Describe alternatives you've considered

- [x] https://github.com/apache/datafusion/issues/13726
- [x] https://github.com/apache/datafusion/issues/13816
- [x] https://github.com/apache/datafusion/issues/13846
- [x] https://github.com/apache/datafusion/issues/13845
- [ ] https://github.com/apache/datafusion/issues/11105
- [x] https://github.com/apache/datafusion/issues/13814
- [ ] https://github.com/apache/datafusion/issues/21048

### Additional context

_No response_

Start by reviewing the linked checklist issues, especially the unchecked #11105 and #21048, along with the recent CI checks referenced in the issue. Define one narrowly scoped build, test, or binary-size improvement before investigating the relevant Cargo configuration or test entry points. Done means the selected bottleneck has a measurable improvement and its tests or CI validation pass.

Written by the indexing model from the issue text.