Ways to scale topic prediction to billions of unseen documents

Nobody has claimed this yet.

Hello! I'm mostly just looking for a sanity check :-)

I would like to train BERTopic over a small subset (1M) of the documents I have and then have it predict on the entire dataset (1B). I already have the embeddings, so I'm just looking for ways to turn them into topics as fast as possible.

1) The cuML implementation is still missing an `.approximate_predict()` function, so we can't use their HDBSCAN model.
2) We can however use cuML's UMAP along with SKLearn's HDBSCAN.

With this approach (mixed cuML UMAP + SKLearn HDBSCAN) I can train the model on 600k documents in 13 minutes (770 doc/s) and then infer on the same batch in 6 minutes (1600 doc/s).

cuML UMAP is only using the GPU for a short amount of time to process the embeddings (think 20 seconds), and everything else is single-threaded, so it should scale ok-ish on multicore machines with a single GPU (I believe there is no reason why a single machine with multiple instances of BERTopic shouldn't do 10k+ second, which is close to a billion per day).

What I'm wondering is, are there better ways of doing this? Like should I use the topic predictions from fitting BERTopic to train a classifier on top of my own embeddings?

If none of this sounds extremely silly I'll go ahead and try to package BERTopic with Triton as an inference server and see how many errors I get :-)

The issue names no repository files or tests. Start by reviewing BERTopic's existing fitting and prediction flow alongside the cuML UMAP and HDBSCAN approach described here, then benchmark a proposed inference path against the stated 1M and 1B-document targets; a validated, documented approach or implementation would define done.

Written by the indexing model from the issue text.