[DISCUSSION] Add support for Spark Connect protocol

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

with spark connect getting popularity should we consider adding support for it?

**Describe the solution you'd like**

although spark connect fits well with ballista architecture this work could be shared with datafusion as well so users can implement non distributed execution should they wish

**Describe alternatives you've considered**

**Additional context**

- https://issues.apache.org/jira/browse/SPARK-58455
- https://github.com/apache/datafusion-comet/issues/4796
- https://spark.apache.org/docs/latest/spark-connect-overview.html

The issue names no implementation files; start with the Spark Connect overview and the linked SPARK-58455 and DataFusion Comet discussions. First determine whether Ballista and DataFusion should share this protocol support and what the integration boundary would be. Done is an agreed scope and implementation plan, since the issue does not define a concrete change.

Written by the indexing model from the issue text.