Should we rewrite/optimize the HNSW graph in 2nd pass?

I was talking to @msokolov just now and this random idea came up... this is just brainstormy/speculation idea at this point:

The HNSW graph is built incrementally: for each new vector to insert, we run a topK search, only seeing all previously added vectors, then add a new node connecting to those top K (we might also prune existing nodes connections to keep diversity). We do this one by one (well, concurrently) ... but this means (maybe?) vectors added early can have poor-ish neighbors since they didn't see all the future vectors yet. But, when future vectors are added, when they are close to an already-added vector, they will also add the return transition (from old node to new node), so this sort of makes up for that old node not seeing future vectors yet?

Anyway, it made me wonder whether a 2nd phase through all the vectors, with the "benefit of hindsight", might somehow yield improvements (smaller graph, faster/better search-time recall curve)?

I think @msokolov mentioned that the original HNSW paper had talked about this behavior but it was somehow OK / a benefit / not a problem?

If we were to plot "average node neighbor distance" as a function of when the node was added, I wonder if we'd see a correlation of lower quality neighbors for earlier added vectors? Or, is the bidirectional linking completely offsetting this bias?

The issue does not name files, tests, or an entry point. Start by locating Lucene's HNSW graph construction and reviewing the original HNSW paper, then measure neighbor distance and search recall by insertion order. Done would require evidence that a second-pass rewrite improves graph size or search quality and a settled implementation plan.

Written by the indexing model from the issue text.