Default num_partitions inconsistent and maybe not correct for large scales

When creating a vector index, if the user does not specify the number of partitions, we have some inconsistent logic to determine the value.

In rust we hit recommended_num_partitions:

pub fn recommended_num_partitions(num_rows: usize, target_partition_size: usize) -> usize {
    // The maximum number of partitions is 4096 to avoid slow KMeans clustering,
    // bump it once we have better clustering algorithms.
    const MAX_PARTITIONS: usize = 4096;
    (num_rows / target_partition_size).clamp(1, MAX_PARTITIONS)
}

In python we hit _target_partition_size_to_num_partitions:

def _target_partition_size_to_num_partitions(
    num_rows: int, target_partition_size: Optional[int]
) -> int:
    if target_partition_size is None:
        target_partition_size = 8192
    num_partitions = num_rows // target_partition_size
    return max(1, num_partitions, 4096)

The first one clamps to 4096. The second one does not (perhaps it was meant to be max(1, min(num_partitions, 4096))?

Furthermore, silently clamping to 4096 is unlikely to provide good results at high scales. We should either warn that the user needs to think about this or use a better default (e.g. sqrt(num_partitions))

Start by comparing Rust's recommended_num_partitions with Python's _target_partition_size_to_num_partitions, including their behavior when the target size is omitted and when row counts are large. Decide and document the intended cap or scaling strategy, then make both paths consistent and verify the high-scale behavior with the relevant vector-index tests.