MaartenGr / MaartenGr/BERTopic

Add GNN message passing and feature aggregation layer for sbert embeddings

Open
#1,065 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

## Summary:
Message passing and feature aggregation are effective techniques for improving the quality of topic clusters in a graph-based topic modeling system. Message passing involves propagating information through the edges of a graph using matrix exponents, which allows information to be shared between nodes and helps to capture the relationships between them. This allows for more accurate modeling of topic clusters and helps to identify hidden themes that may not be apparent in the raw data. By combining these two techniques, the topic model is able to identify more coherent and meaningful topic clusters and produces results that are more informative and useful for downstream analysis.

## Implementation
I've been using message passing and feature aggregation to improve representations for instruction data and then using BERTopic to visualize the data. I've noticed general improvements in topic representation for instructions.
Examples: https://github.com/danielpatrickhug/Sentence_Kernels
Open source implementation: https://github.com/LAION-AI/Open-Assistant/pull/1650

Another potential issue: large topic clusters summarized by chatgpt have brittle topic labels. I would like to add a new summarizer that summarizes topic summaries of different samples of representative docs. I have code for this as well(in sentence kernels).

I can make a PR, please let me know if you have suggestions! :)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing BERTopic's existing topic-modeling entry points and the linked Sentence_Kernels example and Open Assistant pull request. Before implementation, clarify the intended scope between GNN message passing, feature aggregation, and the proposed summarizer; done should mean an agreed integration with measurable improvements and supporting validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.