More efficient Dictionary / constant encoding for partition values in ListingFileProvider

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

The `ListingFileProvider` after https://github.com/apache/arrow-datafusion/pull/1860 uses `UInt16` for the indexes of the `DictionaryArray` type

This means that the number of partitions is limited to `2^16` ~ `64K`. It also means when scanning files from a source that have fewer than 256 distinct values (that could have fit in `UInt8`) there is wasted space and time using larger than needed dictionary columns (which will all have the same value).

**Describe the solution you'd like**
Ideally the partition column would be a constant (or a DictionaryArray with `UInt8` indexes) and the various upstream operations would create `DictionaryArrays` with larger index sizes as needed

**Additional context**
SUggested by @rdettai on https://github.com/apache/arrow-datafusion/pull/1860 at https://github.com/apache/arrow-datafusion/pull/1860#issuecomment-1056764995 and https://github.com/apache/arrow-datafusion/pull/1860#issuecomment-1058110737

Start by reading the ListingFileProvider changes in pull request 1860 and the linked discussion about partition-value encoding. Trace how partition columns are built and how upstream operations handle DictionaryArrays. Done means avoiding the fixed UInt16 limitation while selecting a suitably sized encoding without breaking partition scanning.

Written by the indexing model from the issue text.