MaartenGr / MaartenGr/BERTopic
How to get the respective topics, the name of each topic, the top n words of each topic and other data for news docs on which `transform()` is used?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
Suppose I trained the model first and got the topics, representative docs etc of the training docs using .get_document_info():
topic_model = BERTopic(vectorizer_model=vectorizer_model, hdbscan_model=hdbscan_model, embedding_model=embedding_model)
topics, probs = topic_model.fit_transform(docs)
print(topic_model.get_document_info())
and now I am predicting topics over new_docs:
new_topics, new_probs = topic_model.transform(new_docs)
now how do I get the information which new_doc falls in which new_topic? Like how can I generate a list/df just same as .get_document_info() for the newer docs and its new topics?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the documented get_document_info(), fit_transform(), and transform() entry points, comparing the information returned for training documents with the topic assignments returned for new_docs. Check the API or usage documentation for a supported way to present transformed documents, and consider the work done when the behavior or limitation is clearly documented with an equivalent result for new_docs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100