MaartenGr / MaartenGr/BERTopic

Zero topic distributions for some documents using approximate_distribution()

Open
#2,150 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

### Have you searched existing issues? 🔎

- [X] I have searched and found no existing issues

### Desribe the bug

Dear creators of BERTopic,
Thanks for your work and this package is amazing. I have been using it for a long time. However, I found some documents (no matter whether they are used to train the model) have zero topic distributions for all topics created by BERTopic after applying `approximate_distribution()` function on them. It means that the topic distribution matrix produced by `approximate_distribution()` has some rows having sum of 0. Codes below did several things: (1) build a simple setup for a BERTopic model with PCA and KMEANS (from `cuML`) as the dimension reduction and clustering technique. (2) define a splitting function to split documents and pre-caculated embeddings. (3) fit the model on training data and compute topic distributions for both training and testing data set.

If more information is needed, please let me know. Thanks!

### Reproduction

```python
def pk(num_cluster):
embedding_model = SentenceTransformer('all-MiniLM-L6-v2')
umap_model = PCA(n_components = 10)
hdbscan_model = KMeans(n_clusters = num_cluster)
vectorizer_model = CountVectorizer()
Topic_model = BERTopic(embedding_model=embedding_model, umap_model=umap_model, hdbscan_model=hdbscan_model, vectorizer_model=vectorizer_model,
calculate_probabilities = False,verbose = True)
return Topic_model
def tr_te_split(documents,df,embeddings, i=1):
indices = np.arange(len(documents))
tr_ind, te_ind = train_test_split(indices, test_size=0.2, shuffle= True, random_state=i)
tr_df = df.iloc[tr_ind,:]
te_df = df.iloc[te_ind,:]
tr_documents = [documents[ind] for ind in tr_ind]
te_documents = [documents[ind] for ind in te_ind]
tr_embeddings = embeddings[tr_ind,:]
return tr_df,te_df,tr_documents,te_documents,tr_embeddings
def check_zero_exposure(arr):
if 0 in np.apply_along_axis(arr=arr,func1d=np.sum,axis=1):
return 1
zero_exposures = {}
for year in year_list:
df = pd.read_csv(df_folder+f"/contem_{year}_senti.csv")
documents = df.documents.tolist()
embeddings = np.load(embeddings_folder+f"/contem_{year}_senti_embeddings.npy")
tr_df, te_df, tr_documents,te_documents,tr_embeddings = tr_te_split(documents,df,embeddings)
tr_df.reset_index(drop=True,inplace=True)
te_df.reset_index(drop=True,inplace=True)
topic_model = pk(cluster_num)
topic_model.fit(tr_documents,tr_embeddings)
tr_topic_dist, _ = topic_model.approximate_distribution(tr_documents)
te_topic_dist, _ = topic_model.approximate_distribution(te_documents)
zero_exposure = [check_zero_exposure(tr_topic_dist),check_zero_exposure(te_topic_dist)]
zero_exposures[year] = zero_exposure

# zero_exposures
# {2014: [1, 1],
# 2015: [1, 1],
# 2016: [1, 1],
# 2017: [1, 1],
# 2018: [1, 1],
# 2019: [1, 1],
# 2020: [1, 1],
# 2021: [1, 1],
# 2022: [1, 1],
# 2023: [1, 1]}
```

### BERTopic Version

0.16.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source files or tests are named; start by locating approximate_distribution() and running the supplied BERTopic 0.16.2 reproduction with the PCA and KMeans setup. Trace how document-topic rows are produced and add a regression check for the reported zero-sum rows, with completion defined by the reproduction no longer producing them.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.