MaartenGr / MaartenGr/BERTopic

Loaded Model wont .fit or fit and transform

Open
#1,584 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

Hello Maarten, I loved getting to learn BERTopic for a recent project, I got my model to work properly, and using this code

embedding_model = "sentence-transformers/all-MiniLM-L6-v2"
stored_models_path = "./Stored_Models/"
model.save(stored_models_path, serialization="pytorch", save_ctfidf=True, save_embedding_model=embedding_model)

I saved it however when I run

topics, probability = loaded_model.fit(doc2)
or just loaded_model.fit(doc2)

I get this erorr

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

Do you know what I could be doing wrong? when I run the loaded_model cell I get a value of

so I think it saved properly. (I have also tried the pickling method with no luck)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure using the shown model.save call, a reload into loaded_model, and loaded_model.fit(doc2). Start at the BERTopic save/load and fit entry points; done means a saved and reloaded model can fit doc2 without the reported array-shape error, with the expected return values.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.