Returning fewer topics than specified in nr_topics despite ample data

Hi @MaartenGr,

I must admit that this is a fairly new issue for me as previously our pipeline seemed to be producing outputs as expected. However, recently when I fit a topic model to my data and embeddings it only returns a small number of topics, with the vast majority of records being assigned to topic 1.

In other words, with an input of ~40k docs, and a specified nr_topics=20, it returns a -1, 0, and 1 topic, with about 90% of the data in topic 0.

The output of topic_mode.get_params() returns the following:

``` bertopic params: {
"calculate_probabilities": False,
"ctfidf_model": ClassTfidfTransformer(),
"embedding_model": None,
"hdbscan_model": HDBSCAN(min_cluster_size=785, min_samples=560),
"language": "multilingual",
"low_memory": False,
"min_topic_size": 5,
"n_gram_range": (1, 2),
"nr_topics": 20,
"representation_model": None,
"seed_topic_list": None,
"top_n_words": 10,
"umap_model": UMAP(
min_dist=0.020396346954772607,
tqdm_kwds={
"bar_format": "{desc}: {percentage:3.0f}%| {bar} {n_fmt}/{total_fmt} [{elapsed}]",
"desc": "Epochs completed",
"disable": True,
},
),
"vectorizer_model": CountVectorizer(ngram_range=(1, 2), stop_words="english"),
"verbose": False,
}
```

Which looks pretty legitimate to me. I'm struggling to see why that combination of parameters would return a small/imbalanced number of topics, or why it would return less than the specified 20 topics.

Any insight you might be able to provide would be greatly appreciated.

Many thanks, H

Start by reproducing the BERTopic fit with the reported ~40k documents, embeddings, and get_params() configuration, focusing on the HDBSCAN and UMAP settings. Compare the resulting topic counts and assignments with nr_topics=20; done means explaining why only -1, 0, and 1 appear and identifying whether the behavior comes from the data or these parameters.

Written by the indexing model from the issue text.