Question on incremental training and a suggestion

Nobody has claimed this yet.

Hi, Thank you so much for creating such a cool and interactive and highly modular technique in Topic Modeling.

I have a question and a recommendation.

Regarding the question, I'm curious if there are any options for online training while still using HDBSCAN and UMAP. These methods offer powerful concepts for dimensionality reduction and hierarchical density-based clustering, and I would like to retain their benefits. Other methods tend to lack support for noise detection since they don't include a density component. I'm interested in your suggestions for online training that can preserve these outstanding components.

Additionally, I'd like to recommend using the following function to wrap the document text for hovertext:

def insert_newlines(text, every=20):
words = text.split()
lines = [' '.join(words[i:i+every]) for i in range(0, len(words), every)]
return '
'.join(lines)

By applying this function to selection["doc"], the document text can be made more readable with hovertext in document plots.
I've tested the function with every = 20, and it works effectively with documents of any length.

Start by reviewing BERTopic's online-training options involving UMAP and HDBSCAN, then inspect the document-plot code where selection["doc"] is used for hovertext. The issue does not name files or tests. Done would require a decided approach for incremental training and clearer hovertext, with project tests or examples updated accordingly.

Written by the indexing model from the issue text.