Cannot reproduce the same result

@MaartenGr

I encountered the symptom, which I inquired about via a ticket below again.
I realised the ticket was already closed, so opened a new ticket here.

https://github.com/MaartenGr/BERTopic/issues/275#issuecomment-1505170008

I got different results whenever it was executed. Once it looked resolved by the code below.

• Delete nr_topics
• Delete n_components

However, I executed it again today, and then my model could not reproduce the same results.
So, I would like to know how to configure it to get the same result always.

from bertopic import BERTopic
from sentence_transformers import SentenceTransformer, util
from umap import UMAP
from hdbscan import HDBSCAN
from sklearn.feature_extraction.text import CountVectorizer
from bertopic.vectorizers import ClassTfidfTransformer
from sklearn.cluster import KMeans

import nltk
from nltk.corpus import stopwords
nltk.download("punkt")
nltk.download("wordnet")
nltk.download("omw-1.4")
nltk.download("english")
nltk.download("stopwords")

"""
Reference URL: https://github.com/MaartenGr/BERTopic/issues/286
"""
from nltk import word_tokenize          
from nltk.stem import WordNetLemmatizer

class LemmaTokenizer:
  def __init__(self):
    self.wnl = WordNetLemmatizer()
  def __call__(self, doc):
    return [self.wnl.lemmatize(t) for t in word_tokenize(doc)]

n_neighbors = 30
min_cluster_size = 25
top_n_words = 100
min_samples = 1
ngram_range = (1, 3)

sentence_model = SentenceTransformer("all-MiniLM-L12-v2")
umap_model = UMAP(n_neighbors=n_neighbors, min_dist=0.0, metric="cosine", random_state=42)
hdbscan_model = HDBSCAN(min_cluster_size=min_cluster_size, metric="euclidean", cluster_selection_method="eom", prediction_data=True, min_samples=min_samples)
ctfidf_model = ClassTfidfTransformer(reduce_frequent_words=True, bm25_weighting=True)
vectorizer_model = CountVectorizer(ngram_range=ngram_range, max_df=0.70, tokenizer=LemmaTokenizer(), stop_words=stopwords.words("english"))
model = BERTopic(language="english", top_n_words=top_n_words,
                       embedding_model=sentence_model, umap_model=umap_model, hdbscan_model=hdbscan_model, 
                       ctfidf_model=ctfidf_model, vectorizer_model=vectorizer_model,
                      calculate_probabilities=True)

topics, probs = model.fit_transform(abstract)

Thank you very much.
Kuniko

Start by reviewing the supplied BERTopic configuration and the earlier linked issue, then reproduce the differing outputs around model.fit_transform(abstract). Inspect the UMAP, HDBSCAN, SentenceTransformer, and scikit-learn components named in the example. Done means documenting or implementing a configuration that reliably produces the same results across repeated executions.