Consensus Clustering

Nobody has claimed this yet.

Consensus Clustering is an important aspect when it comes to using any clustering method in production. I don't want my users facing a set of new clusters each time deploying the pipeline.
You have suggested fixating UMAP random seeds to stabilise the clusters. But in practice how can we trust a single randomly picked random seed, while running it with another random seed generates a set of new clusters, and assigns docs to different clusters? This behaviour even escalates more by changing other UMAP, HDBSCAN parameters.

Since there are no alternative clustering measures for un-supervised topic modeling, shouldn't there exist a Consensus Clustering framework, where we can assign docs that are falling together all the time, to a final cluster?

The issue names no files, tests, or entry points. Start by reviewing how UMAP and HDBSCAN parameters affect the clustering pipeline, then define how consensus across runs should assign documents to final clusters and how that result would be validated.

Written by the indexing model from the issue text.