MaartenGr / MaartenGr/BERTopic

Ordering of MMR in chained representation models has no effect

Open
#2,394 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

### Have you searched existing issues? 🔎

- [x] I have searched and found no existing issues

### Desribe the bug

Maarten, thank you so much for your work on BERTopic.

[Similar to this issue](https://github.com/MaartenGr/BERTopic/issues/1224), I have a question regarding the ordering of representation models in a [chained representation model setup](https://maartengr.github.io/BERTopic/getting_started/representation/representation.html#chain-models). Here I am trying to get the redundancy benefit of the MMR representation, with the keyword power of KeyBERT. However, I am finding that I lose the MMR benefits when I put it in any kind of chain.

MMR alone yields the following topics, for example, with low redundancy but also small stopwords:

```
mmr = MaximalMarginalRelevance(diversity=0.3)
representation_model = [mmr]
```
> Topic 1: {car; bike; my; in; engine}
Topic 4: {encryption; clipper; chip; keys; government}

KeyBERTInspired alone yields the following topics, with higher redundancy and no small stopwords:

```
kbmr = KeyBERTInspired()
representation_model = [kbmr]
```

> Topic 1: {riding; motorcycle; bikes; bike; ride}
Topic 4: {encryption; cryptography; crypto; encrypted; security}

However, if I chain MMR *after*, say, KeyBERTInspired or PartOfSpeech, I lose the redundancy benefits and the results look identical to those obtained using *only the KeyBERTInspired model*:

```
mmr = MaximalMarginalRelevance(diversity=0.3)
kbmr = KeyBERTInspired()
representation_model = [kbmr,mmr]
```

> Topic 1: {riding, motorcycle, bikes, bike, ride}
Topic 4: {encryption; cryptography; crypto; encrypted; security}

If I chain MMR *before* KeyBERTInspired, the results are identical to the above.

```
mmr = MaximalMarginalRelevance(diversity=0.3)
kbmr = KeyBERTInspired()
representation_model = [mmr,kbmr]
```

> Topic 1: {riding, motorcycle, bikes, bike, ride}
Topic 4: {encryption; cryptography; crypto; encrypted; security}

To be honest, I doubt this is a bug: more likely, I'm doing something wrong, but I don't know what. I've tried this with both v16.4 and 17.0. Any help would be appreciated! Thank you again!

### Reproduction

```python
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups
from umap import UMAP
from bertopic.representation import MaximalMarginalRelevance, PartOfSpeech, KeyBERTInspired

docs = fetch_20newsgroups(subset='all', remove=('headers', 'footers', 'quotes'))["data"]

seed_topic_list = [["drug", "cancer", "drugs", "doctor"],
["windows", "drive", "dos", "file"],
["space", "launch", "orbit", "lunar"]]

umap_model = UMAP(n_neighbors=20, n_components=5, min_dist=0.1, metric='cosine', random_state=42)

# Specify the chained represenation model
mmr = MaximalMarginalRelevance(diversity=0.3)
posrm = PartOfSpeech("en_core_web_sm")
kbmr = KeyBERTInspired()

representation_model = [mmr, kbmr]

topic_model = BERTopic(
umap_model=umap_model,
min_topic_size=30,
embedding_model="all-MiniLM-L6-v2",
representation_model = representation_model,
).fit(docs)

topics, probs = topic_model.fit_transform(docs)

fig = topic_model.visualize_barchart()
fig.show()

print(topic_model.get_topic_info())

```

### BERTopic Version

0.17

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Python reproduction with BERTopic 0.17 and compare the outputs for the listed representation_model orderings. Read the chained representation-model documentation and the MaximalMarginalRelevance and KeyBERTInspired entry points to determine whether ordering is expected to matter. Done means clarifying or correcting the observed behavior and adding or updating coverage for the ordering case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.