MaartenGr / MaartenGr/BERTopic
Saving Model to generate topics continuously
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.8k
- Forks
- 920
- Avg merge
- 22h 24m
- Merged PRs (30d)
- 5
Description
Hi, I am trying to generate topics continuously over year and see the trend. Since all years of data cannot be loaded at once, I am thinking to write a for loop that load 1 year data first, run the model and generate the topics, then save the model and topics, clear up memory, and grab another year of data to run it again. I am wondering whether BERTopic model can memorize the topics generated previously and build upon that or does it treat the new year of data as independent and only generate new topics? If latter is the case, how do you recommend me to do? Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by tracing the public BERTopic fitting and model-persistence APIs, then determine whether yearly batches can preserve and extend topics. Done means a documented, reproducible answer or an agreed implementation scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100