MaartenGr / MaartenGr/BERTopic
Add GNN message passing and feature aggregation layer for sbert embeddings
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
## Summary:
Message passing and feature aggregation are effective techniques for improving the quality of topic clusters in a graph-based topic modeling system. Message passing involves propagating information through the edges of a graph using matrix exponents, which allows information to be shared between nodes and helps to capture the relationships between them. This allows for more accurate modeling of topic clusters and helps to identify hidden themes that may not be apparent in the raw data. By combining these two techniques, the topic model is able to identify more coherent and meaningful topic clusters and produces results that are more informative and useful for downstream analysis.
## Implementation
I've been using message passing and feature aggregation to improve representations for instruction data and then using BERTopic to visualize the data. I've noticed general improvements in topic representation for instructions.
Examples: https://github.com/danielpatrickhug/Sentence_Kernels
Open source implementation: https://github.com/LAION-AI/Open-Assistant/pull/1650
Another potential issue: large topic clusters summarized by chatgpt have brittle topic labels. I would like to add a new summarizer that summarizes topic summaries of different samples of representative docs. I have code for this as well(in sentence kernels).
I can make a PR, please let me know if you have suggestions! :)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing BERTopic's existing topic-modeling entry points and the linked Sentence_Kernels example and Open Assistant pull request. Before implementation, clarify the intended scope between GNN message passing, feature aggregation, and the proposed summarizer; done should mean an agreed integration with measurable improvements and supporting validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100