Add Local Scripts to Reproduce Full CI

### Is your feature request related to a problem or challenge?

It would be great to have a local script to run full CI, for all functional and non-functional(e.g. lints) tests, this helps development process for human and AI agents.

```
./dev/ci.sh # Run full CI locally
```

Here is a previous work that provides a single script to run some non-functional tests
- https://github.com/apache/datafusion/issues/19227

The remaining work is to
- Continue supporting more non-functional tests
- Create a similar script for all functional tests (rust tests, sqllogictests, extended tests, ...)

(might use xtask instead of scripts, see #23414)
~~### Tracker for non-functional tests~~

`./dev/rust_lint.sh` should fully reproduce non-functional CI locally.
`./dev/rust_lint.sh --write --allow-dirty` should do best-effort fixing where applicable.

Today the script covers cargo fmt, clippy, Cargo.toml formatting, license headers, typos, markdown prettier, and cargo doc.

Remaining CI checks that could be migrated similarly:

* [ ] Semver / breaking API detector: `.github/workflows/breaking_changes_detector.yml`, `ci/scripts/changed_crates.sh`
* [ ] Markdown link check: `.github/workflows/dev.yml`, `ci/scripts/markdown_link_check.sh`
* [ ] ASF required status check validation: `.github/workflows/dev.yml`, `ci/scripts/check_asf_yaml_status_checks.py`
* [ ] Security audit: `.github/workflows/audit.yml`, cargo audit
* [ ] Dependency checks: `.github/workflows/dependencies.yml`, `dev/depcheck`, cargo machete
* [ ] Large file check: `.github/workflows/large_files.yml`
* [ ] CodeQL for GitHub Actions: `.github/workflows/codeql.yml`
* [ ] Docs HTML build: `.github/workflows/docs_pr.yaml`, `docs/build.sh`
* [ ] Config/function docs freshness: `.github/workflows/rust.yml`, `dev/update_config_docs.sh`, `dev/update_function_docs.sh`
* [ ] Examples README freshness: `.github/workflows/rust.yml`, `ci/scripts/check_examples_docs.sh`

### Potential Improvements to `rust_lint.sh`

* [ ] Better naming.
* [ ] Support running subsets, so developers and AI agents can run only checks likely affected by local changes.

Start with dev/rust_lint.sh and compare it with the referenced GitHub workflow files and CI scripts. Identify a bounded group of remaining checks, such as dependency or documentation checks, and determine how the local entry point should invoke them. Done means the selected checks run locally with behavior matching CI, including any documented fixing or subset options.

Written by the indexing model from the issue text.