MaartenGr / MaartenGr/BERTopic

Guided topic modelling np.average function not behaving as expected?

Open
#1,799 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.8k
Forks
920
Avg merge
22h 24m
Merged PRs (30d)
5

Description

## Issue:
`ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.` encountered during model fitting with provided seed words for guided topic modelling

### Description
A `ValueError` was encountered when attempting to fit a topic model using BERTopic with the following configuration:
```python
# Doc is a corpus of about 3K posts
doc = df[df['post_text_clean'] != '']['post_text_clean'].tolist()

sentence_model = SentenceTransformer("all-MiniLM-L6-v2")
embedding = sentence_model.encode(doc, show_progress_bar=True)

seed_topic_list = [
['surgery', 'bottom', 'top', 'hormone'],
['accept', 'acceptance', 'strength', 'needs'],
['connectedness', 'support', 'activism', 'mentor'],
['stopped', 'cancelled', 'pass', 'confusion'],
['peers', 'family', 'friends', 'group'],
['anxiety', 'depression', 'dissociation', 'anorexia'],
['dysphoria', 'familial', 'stress', 'health'],
['impulsive', 'introverted', 'sensitivity', 'shame'],
['violence', 'rejection', 'victimization', 'affirmation'],
['ideation', 'attempt', 'risk', 'prevention']
]

vectorizer_model = CountVectorizer(stop_words = 'english')
ctfidf_model = ClassTfidfTransformer(reduce_frequent_words=True)

main_representation_model = KeyBERTInspired()
aspect_representation_model1 = PartOfSpeech("en_core_web_sm")
aspect_representation_model2 = [KeyBERTInspired(top_n_words=30),
MaximalMarginalRelevance(diversity=.5)]

topic_model = BERTopic(embedding_model=sentence_model,
calculate_probabilities=True,
vectorizer_model = vectorizer_model,
ctfidf_model=ctfidf_model,
representation_model = representation_model,
seed_topic_list=seed_topic_list
)

topic, probs = topic_model.fit_transform(doc, embedding)
```
The error occurs when calling the `fit_transform` method on a BERTopic instance with a set of documents and their embeddings.

Probably the internal call to `np.average` is not behaving as expected?

When attempting to use np.average to compute a weighted average of document embeddings and seed topic embeddings, the ValueError is encountered due to passing a list of arrays with different shapes to np.average, leading to an inhomogeneous shape.
This should update the document embeddings with a weighted influence from corresponding seed topic embeddings.

### Steps to Reproduce
1. Installed numpy version: 1.25.0
2. Initialize BERTopic model with guided modelling approach.
3. Prepare a dataset of documents and their corresponding embeddings.
4. Call the `fit_transform` method on the BERTopic model.

### Error Traceback
```python
ValueError Traceback (most recent call last)
Cell In[7], line 104
102 # Topic Model Fitting
103 print("Topic model fitting..")
--> 104 topic, probs = topic_model.fit_transform(doc, embedding)
106 # Save Model State Checkpoint
107 print("Saving model embeddings checkpoint..")

File c:\Users\georg\anaconda3\Lib\site-packages\bertopic\_bertopic.py:399, in BERTopic.fit_transform(self, documents, embeddings, images, y)
397 # Guided Topic Modeling
398 if self.seed_topic_list is not None and self.embedding_model is not None:
--> 399 y, embeddings = self._guided_topic_modeling(embeddings)
401 # Zero-shot Topic Modeling
402 if self._is_zeroshot():

File c:\Users\georg\anaconda3\Lib\site-packages\bertopic\_bertopic.py:3617, in BERTopic._guided_topic_modeling(self, embeddings)
3615 for seed_topic in range(len(seed_topic_list)):
3616 indices = [index for index, topic in enumerate(y) if topic == seed_topic]
-> 3617 embeddings[indices] = np.average([embeddings[indices], seed_topic_embeddings[seed_topic]], weights=[3, 1])
3618 logger.info("Guided - Completed \u2713")

File c:\Users\georg\anaconda3\Lib\site-packages\numpy\lib\function_base.py:511, in average(a, axis, weights, returned, keepdims)
398 @array_function_dispatch(_average_dispatcher)
399 def average(a, axis=None, weights=None, returned=False, *,
400 keepdims=np._NoValue):
401 """
402 Compute the weighted average along the specified axis.
403
(...)
509 [4.5]])
510 """
--> 511 a = np.asanyarray(a)
513 if keepdims is np._NoValue:
514 # Don't pass on the keepdims argument if one wasn't given.
515 keepdims_kw = {}

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in _bertopic.py, in BERTopic.fit_transform and _guided_topic_modeling around the np.average call at line 3617. Reproduce the guided topic modeling example with the supplied documents, embeddings, and seed topics, then inspect the shapes passed to np.average. Done means fit_transform completes without the reported ValueError while applying the intended weighted influence of seed topic embeddings.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.