Opportunities for late materialization in knn_combined style scans

When we fallback to a prefiltered knn_combined scan we always do a full scan of the unindexed data. We could improve this:

• If there is no scalar index we can do a full scan of the filter columns followed by a take of the vector data
• If there is a scalar index then we can materialize the index and do a take of the vector data

Example script:

import lance
import pyarrow as pa
import shutil

shutil.rmtree("/tmp/weston", ignore_errors=True)

ds = lance.write_dataset(pa.table({
    "vec": pa.array([[i, i] for i in range(1000)], pa.list_(pa.float32(), 2)),
    "filt": range(1000)
}), "/tmp/weston")

ds.create_index("vec", num_sub_vectors=2, num_partitions=2, index_type="IVF_PQ")

ds = lance.write_dataset(pa.table({
    "vec": pa.array([[i, i] for i in range(1000)], pa.list_(pa.float32(), 2)),
    "filt": range(1000)
}), "/tmp/weston", mode="append")

print(ds.scanner(filter="filt > 500", prefilter=True, nearest={"q": [5, 5], "k": 10, "column": "vec"}).analyze_plan())