AnswerDotAI / AnswerDotAI/claudette
usage() raises a TypeError with the AmazonBedrock client
- Dominant language
- Jupyter Notebook
- Stars
- 316
- Forks
- 43
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to use claudette with Claude 3.5 Sonnet via Bedrock and I run into this error:
```
chat("Which football team is known as the Lionesses?")
> TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
```
Here's a reprex to reproduce the error.
``` python
import boto3
from anthropic import AnthropicBedrock
from claudette import Chat, Client
session = boto3.Session(profile_name="claudette")
credentials = session.get_credentials()
# Claude 3.5 Sonnet on Bedrock with cross-region inference
model = "us.anthropic.claude-3-5-sonnet-20241022-v2:0"
ab = AnthropicBedrock(
aws_access_key=credentials.access_key,
aws_secret_key=credentials.secret_key,
aws_region="us-west-2",
)
cli = Client(model, cli=ab)
chat = Chat(model, cli=cli, sp="""You are a helpful and concise assistant.""")
chat("Which football team is known as the Lionesses?")
# > return usage(self.input_tokens+b.input_tokens, self.output_tokens+b.output_tokens, getattr(self,'cache_creation_input_tokens',0)+getattr(b,'cache_creation_input_tokens',0), getattr(self,'cache_read_input_tokens',0)+getattr(b,'cache_read_input_tokens',0))
# >
# > TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
```
Here's the requirements.txt. I install these packages in a new environment.
```
fastcore==1.7.28
anthropic==0.42.0
toolslm==0.1.0
msglm==0.0.4
nbdev==2.3.34
boto3==1.35.92
claudette==0.1.1
ipykernel==6.29.5
```
(Unless I'm doing something wrong) the issue seems to be with getting the `cache_creation_input_tokens` and `cache_read_input_tokens` attributes: these are actually None rather than missing. So `getattr(b,'cache_creation_input_tokens',0)` returns None rather than 0.
Something like the following fixes the issue. (It's verbose though and there's probably a better solution; `__repr__` and `total` need fixing as well.)
``` python
@patch
def __add__(self: Usage, b: Usage):
"Add together each of `input_tokens` and `output_tokens`"
cache_creation_input_tokens = getattr(self, "cache_creation_input_tokens", 0) or 0
cache_creation_input_tokens += getattr(b, "cache_creation_input_tokens", 0) or 0
cache_read_input_tokens = getattr(self, "cache_read_input_tokens", 0) or 0
cache_read_input_tokens += getattr(b, "cache_read_input_tokens", 0) or 0
return usage(
self.input_tokens + b.input_tokens,
self.output_tokens + b.output_tokens,
cache_creation_input_tokens,
cache_read_input_tokens,
)
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the Usage.__add__ implementation referenced in the traceback and inspect usage, __repr__, and total handling of optional cache token attributes. Reproduce the supplied AnthropicBedrock and Claude 3.5 Sonnet example, then verify aggregation works when those attributes are None, including the displayed representation and total.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100