[C++][Skyhook] Refactor Skyhook file format to use a higher level interface

### Describe the enhancement requested

The enhancement is to refactor the Skyhook file format (a custom target of the Arrow dataset API) to loosen the coupling with the Arrow dataset API.

The request was originally made via the mailing list ([link][ml-refactor-skyhook]) and mentions that improvements to the dataset API are not possible while maintaining the Skyhook file format in its current form. A reference is also made to Acero and its usage of substrait, perhaps to contextualize a preferred approach.

[ml-refactor-skyhook]: https://lists.apache.org/thread/77jkh04d06brs6l0j1vqzgx5yncp69g1

### Component(s)

C++