Representation on very large documents with LLMs cuts out its own prompts issue

Nobody has claimed this yet.

Hello Maarten!

Love the module. Currently using BERTopic for multilingual data for my thesis. However, I seek the assistance in making LLM works for representation - I somehow got it to work, but still not quite as effective?

My aim is to use an LLM for representation of KEYWORDS and REPRESENTATIVE DOCUMENTS, but these are very very long. I followed your guide on your official page, but couldn't make it truncate, nor fix the length. All documents are articles, and are in 6 separate languages.
For my embeddings, I split the text into chunks and averaged for the intial (custom) embedding stage.

So I used this Zephyr7b approach with LlamaCPP, and it almost worked by not quite... It somehow cuts the representation and names for the topics, but still gives a bit of valuable insight. I also tried with MISTRAL, but this did not work.
And I can't understand how to use a custom prompt in this scenario, compared to what [your guide shows](https://maartengr.github.io/BERTopic/getting_started/representation/llm.html#transformers) with prompt etc

**Code (edit, added more 10:39)**
```python

# Filtered data to test
first_40_rows = combined_df.head(40)

# Extracting 'text' column and converting to list
docs = first_40_rows['cleaned_text'].tolist()

# Extracting custom averaged over 'embeddings' created by ROBERTA and stacking them vertically
embeddings = np.vstack(first_40_rows['embeddings'].to_numpy())

pip install llama-cpp-python --quiet

from bertopic import BERTopic
from bertopic.representation import LlamaCPP

# Use llama.cpp to load in a 4-bit quantized version of Zephyr 7B Alpha
representation_model = LlamaCPP(
"/work/Master/zephyr-7b-alpha.Q4_K_M.gguf"
)

# Use Danish stopwords with CountVectorizer
vectorizer_model = CountVectorizer(stop_words=danish_stopwords,ngram_range=(1, 3), min_df=2)

# Remaining setup
umap_model = UMAP(n_neighbors=15, n_components=5, min_dist=0.0, metric='cosine', random_state=42)
hdbscan_model = HDBSCAN(min_cluster_size=2, metric='euclidean', cluster_selection_method='eom', prediction_data=True)
ctfidf_model = ClassTfidfTransformer()

# Initialize BERTopic with custom models and the updated vectorizer
topic_model = BERTopic(embedding_model=model,
umap_model=umap_model,
hdbscan_model=hdbscan_model,
vectorizer_model=vectorizer_model,
ctfidf_model=ctfidf_model,
verbose=True,
top_n_words=20,
representation_model=representation_model

# Fit the BERTopic model
topics, probs=topic_model.fit_transform(docs, embeddings)

df_topic['Representation'][1]
['\n"Israeli attacks on civilian Palestinians in Gaza and Leb',
'',
'',
'',
'',
'',
'',
'',
'',
'']

df_topic['Representation'][2]
[' "Israeli attacks on Gazan children"\n\nQ: Can you',
'',
'',
'',
'',
'',
'',
'',
'',
'']
```

Start with the BERTopic representation_model=LlamaCPP setup and the fit_transform call shown in the issue, focusing on how long documents and custom prompts are passed to the LLM representation step. Reproduce the truncated topic representations with the Zephyr7b setup, then establish whether prompt configuration or input length handling is responsible. Done means long multilingual documents produce complete, correctly formatted keyword and representative-document representations.

Written by the indexing model from the issue text.