ANN Index explain and analyze

Goal: give users easy way to understand how ANN index can and will perform.

Explain index

Goal: give user an idea of how an ANN index will perform before training the index.

fn explain_index(
    field: &arrow_schema::Field,
    parameters: &dyn VectorIndexParams,
) -> ExplainIndexResults { ... }

struct ExplainIndexResults {
    /// Expected size of the index in bytes
    expected_size_bytes: u64,
    /// Ratio of uncompressed vector size to index size
    compression_ratio: f32,
    /// Expected top-1 in-sample recall when nprobes == num_partitions. Represents
    /// recall impact from compression alone. This is derived from a formula
    /// based on experimental data.
    /// TODO: formula
    recall_top_1_probe_all: f32,
    /// Expected top-1 in-sample recall for default nprobes (10).
    /// Represents overall recall for default nprobes. Higher nprobes and
    /// refine factor will increase recall.
    recall_top_1_probe_default: f32,
}

Analyze index

Goal: give users a way to measure performance of an existing ANN index

async fn analyze_index(
    index_name: &str,
    nprobes: u32,
    refine_factor: usize,
    recall_top_n: usize
) -> AnalyzeIndexResult { ... }

struct AnalyzeIndexResult {
    /// In sample recall
    recall: f32,
    /// Latency when index cache is cold
    cold_latency: Latencies,
    /// Latency when index cache is warm
    warm_latency: Latencies,
}

/// Query latencies in ms
struct Latencies {
   avg_ms: u64,
   p50_ms: u64,
   p90_ms: u64,
   p99_ms: u64
}

No files or tests are named. Start by locating the Rust ANN index implementation and compare its existing index parameters and query paths with the proposed explain_index and analyze_index signatures; done means the specified size, compression, recall, and cold/warm latency results are available.