Zonemap index reading too many files

We have a dataset with a string column called test_id. The dataset has 30K fragments with each fragment having one test_id only for all rows. A simple query like

filter_expr = "test_id='50EA1PGA6SA001834_1751510023040761'"

# Create scanner with filter and stats
scanner = dataset.scanner(
    filter=filter_expr
)

is scanning > 3k fragments instead of 1 so this query is running very slowly.

[2025-11-03T21:47:06Z DEBUG lance_datafusion::exec] Executing plan:
    LanceRead: uri=xx, projection=[redacted], num_fragments=33010, range_before=None, range_after=None, row_id=false, row_addr=false, full_filter=test_id = Utf8("50EA1PGA6SA001834_1751510023040761"), refine_filter=--
      ScalarIndexQuery: query=[test_id = 50EA1PGA6SA001834_1751510023040761]@test_id_idx

From the trace logs i confirmed that zonemap index was used and the fragment 71 which had this data

[2025-11-03T21:47:08Z TRACE lance::io::exec::filtered_read] Planning 1 ranges (3480 rows) from fragment 71 with filter: Some(BinaryExpr(BinaryExpr { left: Column(Column { relation: None, name: "test_id" }), op: Eq, right: Literal(Utf8("50EA1PGA6SA001834_1751510023040761"), None) }))
was correctly read. However other fragments were read too which don't have that data.