Is there a good way to make max_bytes_per_file used in Stable data storage version

Now, with the Stable data storage version, the max_bytes_per_file option did not affect data file size.

Like the example below, the data file has 9000569 bytes in the file.

import lance
import pyarrow as pa

def generate_large_table():
    data = []
    for i in range(1000000):
        if i % 2 == 0:
            data.append({"name": "Alice", "age": 20})
        else:
            data.append({"name": "Bob", "age": 30})
    table = pa.Table.from_pylist(data)
    return table

table = generate_large_table()

lance.write_dataset(table, "./alice_and_bob.lance", max_bytes_per_file=512)

@westonpace is there a good way to limit the data file size?

Start by running the Python reproduction with Lance and PyArrow, then trace how max_bytes_per_file is handled by the Stable data storage path. Confirm the intended size semantics and verify that the generated data files respect the configured limit.