MaartenGr / MaartenGr/BERTopic
Ordering of MMR in chained representation models has no effect
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
### Have you searched existing issues? 🔎
- [x] I have searched and found no existing issues
### Desribe the bug
Maarten, thank you so much for your work on BERTopic.
[Similar to this issue](https://github.com/MaartenGr/BERTopic/issues/1224), I have a question regarding the ordering of representation models in a [chained representation model setup](https://maartengr.github.io/BERTopic/getting_started/representation/representation.html#chain-models). Here I am trying to get the redundancy benefit of the MMR representation, with the keyword power of KeyBERT. However, I am finding that I lose the MMR benefits when I put it in any kind of chain.
MMR alone yields the following topics, for example, with low redundancy but also small stopwords:
```
mmr = MaximalMarginalRelevance(diversity=0.3)
representation_model = [mmr]
```
> Topic 1: {car; bike; my; in; engine}
Topic 4: {encryption; clipper; chip; keys; government}
KeyBERTInspired alone yields the following topics, with higher redundancy and no small stopwords:
```
kbmr = KeyBERTInspired()
representation_model = [kbmr]
```
> Topic 1: {riding; motorcycle; bikes; bike; ride}
Topic 4: {encryption; cryptography; crypto; encrypted; security}
However, if I chain MMR *after*, say, KeyBERTInspired or PartOfSpeech, I lose the redundancy benefits and the results look identical to those obtained using *only the KeyBERTInspired model*:
```
mmr = MaximalMarginalRelevance(diversity=0.3)
kbmr = KeyBERTInspired()
representation_model = [kbmr,mmr]
```
> Topic 1: {riding, motorcycle, bikes, bike, ride}
Topic 4: {encryption; cryptography; crypto; encrypted; security}
If I chain MMR *before* KeyBERTInspired, the results are identical to the above.
```
mmr = MaximalMarginalRelevance(diversity=0.3)
kbmr = KeyBERTInspired()
representation_model = [mmr,kbmr]
```
> Topic 1: {riding, motorcycle, bikes, bike, ride}
Topic 4: {encryption; cryptography; crypto; encrypted; security}
To be honest, I doubt this is a bug: more likely, I'm doing something wrong, but I don't know what. I've tried this with both v16.4 and 17.0. Any help would be appreciated! Thank you again!
### Reproduction
```python
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups
from umap import UMAP
from bertopic.representation import MaximalMarginalRelevance, PartOfSpeech, KeyBERTInspired
docs = fetch_20newsgroups(subset='all', remove=('headers', 'footers', 'quotes'))["data"]
seed_topic_list = [["drug", "cancer", "drugs", "doctor"],
["windows", "drive", "dos", "file"],
["space", "launch", "orbit", "lunar"]]
umap_model = UMAP(n_neighbors=20, n_components=5, min_dist=0.1, metric='cosine', random_state=42)
# Specify the chained represenation model
mmr = MaximalMarginalRelevance(diversity=0.3)
posrm = PartOfSpeech("en_core_web_sm")
kbmr = KeyBERTInspired()
representation_model = [mmr, kbmr]
topic_model = BERTopic(
umap_model=umap_model,
min_topic_size=30,
embedding_model="all-MiniLM-L6-v2",
representation_model = representation_model,
).fit(docs)
topics, probs = topic_model.fit_transform(docs)
fig = topic_model.visualize_barchart()
fig.show()
print(topic_model.get_topic_info())
```
### BERTopic Version
0.17
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided Python reproduction with BERTopic 0.17 and compare the outputs for the listed representation_model orderings. Read the chained representation-model documentation and the MaximalMarginalRelevance and KeyBERTInspired entry points to determine whether ordering is expected to matter. Done means clarifying or correcting the observed behavior and adding or updating coverage for the ordering case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100