Out of memory with tuner.image_to_text processing 400k images via Multimodal Topic Modeling

Nobody has claimed this yet.

Hey, first of all – many tanks for the great package – especially with the update for multimodality, which I am using it for here.

I have image only dataset that I try to run a topic model on. I pre-calculated the embeddings on the GPU and am now trying to build my complete model. Dimensionality reduction and clustering seems to work, but I do get a out of memory at the representation stage while the algorithm is trying to convert the images to text. Is there any way around that?

Currently, I am running the code on a GPU (T4) with 30 GB of RAM, as it seemed to me that HDBSCAN / UMAP on GPU with CUML is faster than the CPU. With the CPU I could get RAM until like 512 GB, but 1,4 TB still seems way to high? Especially in relation to me only working on a dataset of 440k small images…

Do you have any idea on that?

```
2023-08-10 01:24:34,343 - BERTopic - Reduced dimensionality
2023-08-10 01:26:43,227 - BERTopic - Clustered reduced embeddings
Traceback (most recent call last):
File "/efs/users/groesch/ondemand/buildFModelBasedOnEmbeddingsQuer.py", line 35, in
topics = topic_model.fit_transform(documents=None, embeddings=embeddingsLoaded, images=embeddings_joint_imageslist)
File "/users/groesch/.local/lib/python3.10/site-packages/bertopic/_bertopic.py", line 397, in fit_transform
custom_documents = self._images_to_text(documents, embeddings)
File "/users/groesch/.local/lib/python3.10/site-packages/bertopic/_bertopic.py", line 3140, in _images_to_text
documents = tuner.image_to_text(documents, embeddings)
File "/users/groesch/.local/lib/python3.10/site-packages/bertopic/representation/_visual.py", line 167, in image_to_text
indices = mmr(topic_embedding.reshape(1, -1), embeddings[indices], indices, top_n=top_n, diversity=0.1)
File "/users/groesch/.local/lib/python3.10/site-packages/bertopic/representation/_mmr.py", line 96, in mmr
word_similarity = cosine_similarity(word_embeddings)
File "/opt/apps/hand/anaconda3/202303/lib/python3.10/site-packages/sklearn/metrics/pairwise.py", line 1401, in cosine_similarity
K = safe_sparse_dot(X_normalized, Y_normalized.T, dense_output=dense_output)
File "/opt/apps/hand/anaconda3/202303/lib/python3.10/site-packages/sklearn/utils/extmath.py", line 189, in safe_sparse_dot
ret = a @ b
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 1.40 TiB for an array with shape (438365, 438365) and data type float64
```

Start by tracing BERTopic._bertopic.py::_images_to_text into representation/_visual.py::image_to_text and representation/_mmr.py::mmr, then reproduce the failure with the reported image embeddings. Inspect the scikit-learn cosine-similarity call that attempts to allocate a 1.40 TiB array. Done means the representation stage handles this dataset without the reported out-of-memory allocation, with behavior validated at the reported scale.

Written by the indexing model from the issue text.