MaartenGr / MaartenGr/BERTopic

ZeroShot

Open
#1,775 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

I use ZeroShot to classify my documents.
topic_model_zero1 = BERTopic(
hdbscan_model=hdbscan_model,
embedding_model="thenlper/gte-small",
min_topic_size=15,
zeroshot_topic_list=zeroshot_topic_list,
zeroshot_min_similarity=.75,
representation_model=KeyBERTInspired()
)
topics, probs= topic_model_1.fit_transform(texts_zeroshot)
topics, probs = topic_model_1.transform(texts_zeroshot)
Zero_1_Doc_Info=topic_model_1.get_document_info(texts_zeroshot)
topic_distr, _ = topic_model_1.approximate_distribution(texts_zeroshot)

I can get the following information:
Document | Topic | Name | Representation | Representative_Docs | Top_n_words| Representative_document
The representative document is shown as false and there is no probability distribution for each document.
My boss asked for the topic probability distribution of each document and I cannot figure out how to do it. Please check my code and help me to find the information I need. Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the BERTopic entry points get_document_info and approximate_distribution in the codebase, then compare their outputs for the ZeroShot workflow shown. Verify whether the per-document topic probability distribution is returned or needs to be obtained through a different documented result; done means identifying the correct output for each document.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.