MaartenGr / MaartenGr/BERTopic

Merging topics

Open
#1,588 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

Hey Maarten,

I've further refined my topic model but now I'm noticing a couple of weird issues.

If I merge topics (even a single topic into another single topic) my topic model appears to collapse from 32 topics to about 4 including the outliers. The same thing happens if I 'update_topics()' with the countVectoriser.

Another thing I've noticed is that if I set the min_topic_size hyper-parameter it does nothing even though I'm using HDBSCAN. Is that something you have seen before?

Cheers.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the topic collapse after merging topics and after calling update_topics() with the countVectoriser, then separately check whether min_topic_size affects a model using HDBSCAN. The issue does not name files or tests; done would require isolating the cause of both reported behaviors and confirming the expected topic count and minimum-size behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.