MaartenGr / MaartenGr/BERTopic

bertopic version 0.16.0 - when adding representation model together with zeroshot_topic_list end with failure

Open
#1,963 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
921
Avg merge
22h 24m
Merged PRs (30d)
5

Description

from bertopic import BERTopic

2024-05-02 10:26:56,345 - BERTopic - Zeroshot Step 2 - Completed ✓
2024-05-02 10:26:56,346 - BERTopic - Zeroshot Step 3 - Combining clustered topics with the zeroshot model
KeyError: '-1'
File , line 18
1 from bertopic import BERTopic
3 topic_model = BERTopic(
4
5 # Pipeline models
(...)
15 verbose=True
16 )
---> 18 topics, probs = topic_model.fit_transform(docs, embeddings)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-c320d35e-2ba0-4086-9066-6452698cd8ba/lib/python3.11/site-packages/bertopic/_bertopic.py:3150, in BERTopic.merge_models(cls, models, min_similarity, embedding_model)
3147 merged_topics["topic_labels"][str(new_topic_val)] = selected_topics["topic_labels"][str(new_topic)]
3149 if selected_topics["topic_aspects"]:
-> 3150 merged_topics["topic_aspects"][str(new_topic_val)] = selected_topics["topic_aspects"][str(new_topic)]
3152 # Add new embeddings
3153 new_tensors = tensors[new_topic - selected_topics["_outliers"]]

topic_model = BERTopic(

Pipeline models

embedding_model=embedding_model,
umap_model=umap_model,
hdbscan_model=hdbscan_model,
vectorizer_model=vectorizer_model,
zeroshot_topic_list=zero_shot_topics_list,
zeroshot_min_similarity=.8,
representation_model=representation_model,

Hyperparameters

top_n_words=10,
verbose=True
)

topics, probs = topic_model.fit_transform(docs, embeddings)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with BERTopic.fit_transform and inspect merge_models around _bertopic.py:3150, where the traceback occurs. Reproduce the failure using zeroshot_topic_list together with representation_model and confirm that the same configuration completes without a KeyError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.