MaartenGr / MaartenGr/BERTopic

results of `transform` is differnet from merged topic model `get_topic_info()` output

Open
#2,019 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

_Edit: I have surprisingly missed both the `topics_` atrribute and the `get_document_info()` method. My question is changed a little bit and I am now wondering why the transform is different to the original assignment on training?_

I have just noticed a problem I am having where the outputs from the transform don't match the the counts from the `get_topic_info()` method.

That is that the counts of how many documents in a topic are not consitant.

Here is a Minimum Reproducible example:
```python
from umap import UMAP
from bertopic import BERTopic
from datasets import load_dataset

dataset = load_dataset("CShorten/ML-ArXiv-Papers")["train"]

# Extract abstracts to train on and corresponding titles
abstracts_1 = dataset["abstract"][:500]
abstracts_2 = dataset["abstract"][500:1000]
abstracts_3 = dataset["abstract"][1000:1500]

# Create topic models
umap_model = UMAP(n_neighbors=15, n_components=5, min_dist=0.0, metric='cosine', random_state=42)
topic_model_1 = BERTopic(umap_model=umap_model, min_topic_size=20).fit(abstracts_1)
topic_model_2 = BERTopic(umap_model=umap_model, min_topic_size=20).fit(abstracts_2)
topic_model_3 = BERTopic(umap_model=umap_model, min_topic_size=20).fit(abstracts_3)

# Combine all models into one
merged_model = BERTopic.merge_models([topic_model_1, topic_model_2, topic_model_3])

display(merged_model.get_topic_info())

all_abstracts = pd.DataFrame({'documents': abstracts_1 + abstracts_2 + abstracts_3})
all_abstracts['topic'] = merged_model.transform(all_abstracts['documents'])[0]

display(all_abstracts['topic'].value_counts())
```

![image](https://github.com/MaartenGr/BERTopic/assets/103026808/6b443058-cdfe-49cf-ae5e-249bcae9d9e4)

Here is my workling example

```python
embeddings = all_embeddings['voyageai'].copy()

display(embeddings)

mode_groups = embeddings.groupby('mode')
mode_dfs = [mode_groups.get_group(i).reset_index(drop=True) for i in range(3)]

mode_models = [BERTopic() for _ in mode_dfs]

for model, df in zip(mode_models, mode_dfs):
model.fit_transform(
df['si'],
np.array([np.array(x) for x in df['si_embedding'].to_numpy()])
)
display(model.get_topic_info())

merged_model = BERTopic.merge_models(mode_models, min_similarity=0.9)

display(merged_model.get_topic_info())

embeddings['topic'] = merged_model.transform(embeddings['si'], np.array([np.array(x) for x in embeddings['si_embedding'].to_numpy()]))[0]

embeddings['topic'].value_counts()
```

Output:
![image](https://github.com/MaartenGr/BERTopic/assets/103026808/e2f8cc89-7a12-4cda-8584-364f8f812050)

What am I missing and why can the topic assignment be so different from the merged model and the transformed values. Furthermore am I missing how I should be getting the topics for the original documents?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the discrepancy with BERTopic.merge_models, get_topic_info(), transform(), and the supplied datasets and UMAP setup. Read the model state exposed through topics_ and get_document_info(), then compare the original assignments with transformed assignments for the same documents. Done means the issue explains the differing counts and identifies the supported way to retrieve topics for the original documents.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.