Create tests for spec compliance

We've had some issues with forwards and backwards compatibility with the library. That is, datasets made by one version of the Lance library aren't readable and/or writeable by another version of the library. We are improving that case directly by adding more tests (see: #4416). However, a more general way to see it is two libraries are writing datasets that aren't compatible with the other library. In this case, the two libraries are just different versions of the same, but if someone wrote an independent implementation of the Lance format spec, we could also run into situations where we need to be concerned about compatibility with between the Rust Lance library and some other new library.

We don't have another implementation to test against, but what we can do is create some tests against a mock implementation. This should help us test behaviors like:

1. What happens if a Dataset contains an index type we aren't don't know how to use?
   a. Can we still read the dataset? Can we still make queries (just ignoring the index)?
   b. Can we still write to the dataset?
   c. Can we still call optimize_indices()?
2. What happens if the Lance files contains encoding types we don't recognize?
   a. Can we still read the columns that aren't encoded in some unknown encoding?
3. What happens if the manifest has a WriterVersion we don't recognize? [^1]

We should be able to write a test library in Python that can use the protobuf definitions directly to write spec-valid datasets, that aren't tied to the behavior of the Lance Rust library. That will give us a way to test the dataset-level spec compliance.

[^1]: This came up recently in a fix #5113 and #5111.

Start by reviewing the Lance protobuf definitions and the existing compatibility work in #4416, then examine how the Rust library reads and writes datasets. Build a Python test library that produces spec-valid datasets independently, and cover unknown index types, encodings, and manifest WriterVersion values with the read, query, write, and optimize behaviors described in the issue.