MaartenGr / MaartenGr/BERTopic

[Possible BUG] n_words parameter doesn't update y_label values

Open
#1,858 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

Hello!

I'm trying to use your library to do simple topic modeling on the Yahoo answers topics dataset. Currently, I'm trying out the unsupervised approaches. When visualizing the final topics, the `n_words` parameter seems to only update the bars inside the plot but not the y label values as well.

Here is the code and the final image:
```
from umap import UMAP
from hdbscan import HDBSCAN
from sentence_transformers import SentenceTransformer
from sklearn.cluster import KMeans
from bertopic.representation import MaximalMarginalRelevance
from bertopic import BERTopic
from bertopic.representation import KeyBERTInspired
from datasets import load_dataset
import matplotlib.pyplot as plt

print("Downloading dataset . . .")
data = load_dataset("yahoo_answers_topics", split="test")
text = data.to_pandas()[["best_answer", "topic"]]
num_classes = len(set(text.topic.values))

print("Downloading model . . .")
embedding_model = SentenceTransformer("sentence-transformers/all-mpnet-base-v2")

umap_model = UMAP(
n_neighbors=15, n_components=5, min_dist=0.0, metric="cosine"
)
kmeans_model = KMeans(n_clusters=num_classes)

representation_model = [
KeyBERTInspired(),
MaximalMarginalRelevance(diversity=0.5)
]

topic_model = BERTopic(
embedding_model=embedding_model,
umap_model=umap_model,
hdbscan_model=kmeans_model,
representation_model=representation_model,
verbose=True,
)

print("Finding topics . . .")
topics, probs = topic_model.fit_transform(text.best_answer.values)

fig = topic_model.visualize_barchart(
top_n_topics=num_classes,
n_words=10,
)
fig.write_image("topics_barchart.jpg")
```
![topics_barchart](https://github.com/MaartenGr/BERTopic/assets/94498051/4725b984-6c51-40da-80df-8ea7776952ab)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the BERTopic.visualize_barchart entry point used in the example and trace how n_words controls the bars and y-axis labels. Check whether the labels are built from a separate topic-word selection, then reproduce the reported figure with n_words=10. Done means the displayed labels and bars show the same number of words.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.