What things should be in the lance public API?

Clearly defining a public API for the lance crate can make it easier for us to have stable releases because we can know what to count as a breaking change. It will also make lance easier to use as I think it is awkward at the moment that users need to add imports to lance_io, lance_encoding, etc.

In my head, I think the way that the lance library should work, is that everything that can be done with a Dataset can be done with lance imports alone.

Currently, this means we would need to re-export a few things:

• There are a lot of types we'd need to re-export to support training and creating indices. Should we re-export these or should lance_index be considered a "top-level" package that adds capabilities onto lance?
• I think we should probably re-export arrow_array and arrow_schema as we use those types throughout our public API
• Should we re-export datafusion? There are a few public APIs that create plans or consume datafusion expressions that could benefit from this..
• Should we re-export object_store?
• I think we should examine dataset's API and see if there is anywhere we are depending on minor crates like lance_table, lance_io, lance_core, lance_encoding, lance_linalg, etc. We should re-export rather than consider these top-level crates

Start by reviewing the lance crate's Dataset API and tracing the public types it exposes from lance_index, arrow_array, arrow_schema, datafusion, object_store, and the other listed crates. Done means an agreed public-API boundary, including which types are re-exported or remain top-level packages, with the breaking-change implications recorded.