danielmiessler / danielmiessler/Fabric
[Feature request]: allow to have a parameter to set a value in max_tokens for some models (claude-3-5-sonnet)
- Dominant language
- Go
- Stars
- 43.9k
- Forks
- 4.3k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 8
Description
### What do you need?
Maybe there is a way to change this in a proper way but I couldn't find it.
In my case I had problems with making claude-3-5-sonnet work correctly, I was able to fix it by increasing the number of max_tokens allowed in utils.py
```
async def claudeStream(self, system, user):
from anthropic import AsyncAnthropic.
self.claudeApiKey = os.environ["CLAUDE_API_KEY"]
Streamingclient = AsyncAnthropic(api_key=self.claudeApiKey)
buffer = ""
async with Streamingclient.messages.stream(
max_tokens=40096,
system=system,
messages=[user],
model=self.model, temperature=self.args.temp, top_p=self.args.top_p
```
Contributor guide
Assessment
This issue has not been assessed yet.