A slowly grown index created by calls to optimize does not yield as many partitions as one might expect

I have an index that was created on a small dataset and then the dataset grew over time. I called optimize_indices on the dataset every so often to keep the index up to date. However, the final result did not have as many partitions as I would hope:

9,138,882 rows (should indicate 2231 partitions of size 4Ki)
373 partitions (far fewer than expected)
Of these partitions, more than 50% of the rows were in the top 64 centroids

I think one problem might be that we only split once per call to optimize. If we've added a lot of rows we might need to split several times recursively to get all the resulting partitions into the target size.

Start by tracing the optimize_indices entry point and how it chooses partition splits when a dataset has grown. Reproduce the reported case with a small index that grows over time, then verify that optimization reaches the target partition size and produces the expected number of partitions.