DimitriGilbert / DimitriGilbert/LiteChat
Add limits to context collection to prevent excessive prompt sizes in enhance block
- Dominant language
- TypeScript
- Stars
- 52
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
CodeRabbit
Add limits to context collection to prevent excessive prompt sizes
When collecting context blocks from conversation history, there's no limit on the number of blocks included. This could lead to excessively large prompts that exceed model token limits.
```diff
const contextBlocks = interactionsToScan
.flatMap((inter) =>
typeof inter.response === "string"
? this.parseMarkdownForCodeBlocks(inter.response)
: []
)
.filter((block) => block.code !== originalContent);
+
+ // Limit context to prevent excessive prompt sizes
+ const MAX_CONTEXT_BLOCKS = 5;
+ const limitedContextBlocks = contextBlocks.slice(-MAX_CONTEXT_BLOCKS);
+
- if (contextBlocks.length > 0) {
+ if (limitedContextBlocks.length > 0) {
contextPrompt = `\n\nFor context, here are other code blocks from the ${
mode.includes("conversation") ? "conversation" : "message"
- }:\n\n${contextBlocks
+ }:\n\n${limitedContextBlocks
.map((b) => `\`\`\`${b.lang || ""}\n${b.code}\n\`\`\``)
.join("\n\n")}`;
}
```
why hardcode then...
guess imma have to add another setting
"ooohh yeah, i am going to give all the knobs it will be great" he said... 😢
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.