feat: Support test spark runner in `datafusion-spark` for slt tests

The `datafusion-spark` crate is evolving rapidly and its functionallity covered by integration tests, however it would be awesome to have a local Spark runner which can verify datafusion-spark functions results.

Currently the similar exists in Comet when all the spark internal tests run on the Comet codebase, but this approach barely transferrable.

Would be good to have a separate CI or tool to run local Spark and check `datafusion-spark` functions and Spark ones returns the same result.

_Originally posted by @comphead in https://github.com/apache/datafusion/pull/17032#pullrequestreview-3089272585_