MaartenGr / MaartenGr/BERTopic

Suggestion: make documents optional for BERTopic.transform()

Open
#366 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

Hi Maarten,

Thank you once again for your fantastic work :))

I'm using BERTopic as part of a sklearn-style pipeline. Looking at the transform function, it seems that it only uses the documents-input if embeddings are not provided. Therefore it might make sense to make documents an optional parameter. Right not I use a hacky workaround where I provide it with some sham-documents, but this introduces a bit of overhead for large datasets. Would this be possible or would it disturb the API too much?

I would be more than happy to implement it, but it shouldn't be too difficult (if you think it is a good idea)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading BERTopic's transform function and the path used when embeddings are supplied. Check how documents are currently required in that case and verify the sklearn-style pipeline behavior. Done means callers can omit documents when providing embeddings without the current sham-document overhead, with the existing transform behavior preserved otherwise.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
api, machine-learning
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.