MaartenGr / MaartenGr/BERTopic

How to get the respective topics, the name of each topic, the top n words of each topic and other data for news docs on which `transform()` is used?

Open
#1,832 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

Suppose I trained the model first and got the topics, representative docs etc of the training docs using .get_document_info():

topic_model = BERTopic(vectorizer_model=vectorizer_model, hdbscan_model=hdbscan_model, embedding_model=embedding_model)
topics, probs = topic_model.fit_transform(docs)
print(topic_model.get_document_info())

and now I am predicting topics over new_docs:

new_topics, new_probs = topic_model.transform(new_docs)

now how do I get the information which new_doc falls in which new_topic? Like how can I generate a list/df just same as .get_document_info() for the newer docs and its new topics?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the documented get_document_info(), fit_transform(), and transform() entry points, comparing the information returned for training documents with the topic assignments returned for new_docs. Check the API or usage documentation for a supported way to present transformed documents, and consider the work done when the behavior or limitation is clearly documented with an equivalent result for new_docs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.