[FEA] Explore potential for incremental/online PCA to improve performance of layer0 (often brute-force) index

This could use online PCA or a set of centroids could be trained up front to make online pq possible. The idea here is to reduce the time between vectors being received in a vector database and being fully searchable.

In most cases, we should not have to wait for enough vectors to train a full ANN index and brute-force should be able to satisfy searching in the meantime. As brute-force is exhaustive, we have 2 basic options for being able to improve the performance:

1. We prune distances using an IVF method. Random ball cover could also be a great drop-in replacement for bfknn.

2. We reduce the number of dimensions. Incremental/online PCA could help reduce the concept drift over time while still providing a very fast mechanism for reducing the dimensionality significantly. Since these newly inserted vectors are often shoved in smaller segment files until merged together into a larger index, we could investigate something like random projections or rp-trees, which are great for smaller chunks of data (usually on the order of 1M-10M) and can maintain distance error within a provable epsilon (which is based only on the number of vectors).

The primary goal here is to reduce the time between a vector being sent to a vector db and it being fully searchable. We also need to do this at scale.

No files, tests, or entry points are named. Begin by mapping the layer0 brute-force index and its segment-merge path, then compare the proposed IVF, random-ball-cover, online PCA, random-projection, and RP-tree directions. Done means demonstrating a scalable reduction in the time from vector insertion to full searchability while preserving an acceptable distance error.