MaartenGr / MaartenGr/BERTopic
reduce_outliers and update_topics remove stop_words and ngram_range effects
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
### Have you searched existing issues? 🔎
- [X] I have searched and found no existing issues
### Desribe the bug
After running `reduce_outliers` and `update_topics`, the effects of all specifications used in `vectorizer_model` (stop words, ngram) are gone. The results' representation words only show single words. Thanks.

```
vectorizer_model = CountVectorizer(stop_words=stop_words, ngram_range=(1, 4), min_df=5)
representation_model = MaximalMarginalRelevance(diversity=0.5)
topic_model_outlier_reduction = BERTopic(
vectorizer_model=vectorizer_model,
representation_model=representation_model,
top_n_words=15,
min_topic_size=15,
calculate_probabilities=True
)
topics_outlier_reduction, probs_outlier_reduction = topic_model_outlier_reduction.fit_transform(docs, embeddings)
new_topics = topic_model_outlier_reduction.reduce_outliers(docs,
topics_outlier_reduction,
threshold=0.2, strategy="distributions") # probabilities=probs_outlier_reduction,
topic_model_outlier_reduction.update_topics(docs, topics=new_topics)
```
### BERTopic Version
0.16.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the reduce_outliers and update_topics entry points and trace how the configured CountVectorizer is used after topics are reassigned. Reproduce the example with stop_words and ngram_range=(1, 4); done means updated topic representations preserve those vectorizer effects rather than showing only single words.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, scikit-learn
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100