Updating topic frequencies

Thank you for creating this! I do not come from a coding background and this the first time I have would with a model - BERTopic made it really easy for me to do my analysis and also made it easier to understand how models work.

I got an output after fitting the model, which I then ran through topic distributions. After checking the output of topic distributions, I saw that the topic distributions has a better mapping of the topics to the documents. Eg, document X was assigned to topic 1 in the initial model with a lesser probability - the topic distributions showed that topic 10 is a better match for document X.

Therefore, I want to assign the highest probabilities from the topic distributions array. In order to do this, I extracted the topic list (in the same format as 'topics') with topics assigned to each row in the dataframe - based on the highest probability.

Now, when I am trying to update the topic frequencies using:

documents = pd.DataFrame({"Document": docs, "Topic": topics2})
loaded_model._update_topic_size(documents)

This is the output I get:

I am unable to run any visualizations as well:

Please help me in getting about this issue. Also, is there a way to integrate 'topic distributions' during the process of fitting the model? I feel that may give me better results.

Thanks a lot!

Start with the _update_topic_size(documents) call and the topic distributions output described in the issue, then compare those results with the visualization failures. Determine whether the updated topic assignments are supported and document or correct the behavior so topic frequencies and visualizations work as intended.