MaartenGr / MaartenGr/BERTopic
bertopic version 0.16.0 - probs are empty when executing with zero_shot
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
when executing zero shot. the following probs is empty:
topics, probs = topic_model.fit_transform(docs, embeddings)
following configuration:
from umap import UMAP
umap_model = UMAP(n_neighbors=15, n_components=5, min_dist=0.0, metric='cosine', random_state=42)
from hdbscan import HDBSCAN
hdbscan_model = HDBSCAN(min_cluster_size=200, metric='euclidean', cluster_selection_method='eom', prediction_data=True)
from sklearn.feature_extraction.text import CountVectorizer
vectorizer_model = CountVectorizer(stop_words="english")
zero_shot_topics_list=["vitiligo"]
embedding_model_name="thenlper/gte-base"
embedding_model = SentenceTransformer(embedding_model_name)
embeddings = embedding_model.encode(docs, show_progress_bar=True)
topic_model = BERTopic(
# Pipeline models
embedding_model=embedding_model,
umap_model=umap_model,
hdbscan_model=hdbscan_model,
vectorizer_model = CountVectorizer(stop_words="english")
zeroshot_topic_list=zero_shot_topics_list,
zeroshot_min_similarity=.8,
calculate_probabilities=True,
# representation_model=representation_model,
# Hyperparameters
top_n_words=10,
verbose=True
)
topics, probs = topic_model.fit_transform(docs, embeddings)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported BERTopic 0.16.0 configuration with zero-shot topics, calculate_probabilities=True, and the provided fit_transform call. Trace how zero-shot assignment produces the probs result; done means the call returns the expected probability values instead of an empty result, with regression coverage for this configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100