danielmiessler / danielmiessler/Fabric

[Feature request]: allow to have a parameter to set a value in max_tokens for some models (claude-3-5-sonnet)

Open
#720 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
43.9k
Forks
4.3k
Avg merge
1d 4h
Merged PRs (30d)
8

Description

### What do you need?

Maybe there is a way to change this in a proper way but I couldn't find it.
In my case I had problems with making claude-3-5-sonnet work correctly, I was able to fix it by increasing the number of max_tokens allowed in utils.py

```
async def claudeStream(self, system, user):
from anthropic import AsyncAnthropic.
self.claudeApiKey = os.environ["CLAUDE_API_KEY"]
Streamingclient = AsyncAnthropic(api_key=self.claudeApiKey)
buffer = ""
async with Streamingclient.messages.stream(
max_tokens=40096,
system=system,
messages=[user],
model=self.model, temperature=self.args.temp, top_p=self.args.top_p
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.