DimitriGilbert / DimitriGilbert/LiteChat

Consider making maxTokens configurable based on model capabilities

Open
#69 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
52
Forks
10
PR merge metrics
No merged PRs in 30d

Description

CodeRabbit
Consider making maxTokens configurable based on model capabilities

The hardcoded maxTokens: 4096 might not be suitable for all models. Some models support more, while others support less.

```
+ // Get model-specific max tokens or use a reasonable default
+ const modelMaxTokens = modelInstance.maxOutputTokens || 4096;
+ const maxTokens = Math.min(modelMaxTokens, 4096); // Cap at 4096 for code generation
+
const result = await AIService.generateCompletion({
model: modelInstance,
system: systemPrompt,
messages: [{ role: "user", content: userPrompt }],
temperature: 0.1,
- maxTokens: 4096,
+ maxTokens: maxTokens,
});
```

settings... i might have to ask suno for something...

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.