MaartenGr / MaartenGr/BERTopic
Suggestion: make documents optional for BERTopic.transform()
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
Hi Maarten,
Thank you once again for your fantastic work :))
I'm using BERTopic as part of a sklearn-style pipeline. Looking at the transform function, it seems that it only uses the documents-input if embeddings are not provided. Therefore it might make sense to make documents an optional parameter. Right not I use a hacky workaround where I provide it with some sham-documents, but this introduces a bit of overhead for large datasets. Would this be possible or would it disturb the API too much?
I would be more than happy to implement it, but it shouldn't be too difficult (if you think it is a good idea)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading BERTopic's transform function and the path used when embeddings are supplied. Check how documents are currently required in that case and verify the sklearn-style pipeline behavior. Done means callers can omit documents when providing embeddings without the current sham-document overhead, with the existing transform behavior preserved otherwise.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, scikit-learn
- Domain
- api, machine-learning
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100