AnswerDotAI / AnswerDotAI/claudette

usage() raises a TypeError with the AmazonBedrock client

Open
#50 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
316
Forks
43
PR merge metrics
No merged PRs in 30d

Description

I'm trying to use claudette with Claude 3.5 Sonnet via Bedrock and I run into this error:
```
chat("Which football team is known as the Lionesses?")
> TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
```
Here's a reprex to reproduce the error.
``` python
import boto3
from anthropic import AnthropicBedrock
from claudette import Chat, Client

session = boto3.Session(profile_name="claudette")
credentials = session.get_credentials()

# Claude 3.5 Sonnet on Bedrock with cross-region inference
model = "us.anthropic.claude-3-5-sonnet-20241022-v2:0"

ab = AnthropicBedrock(
aws_access_key=credentials.access_key,
aws_secret_key=credentials.secret_key,
aws_region="us-west-2",
)
cli = Client(model, cli=ab)
chat = Chat(model, cli=cli, sp="""You are a helpful and concise assistant.""")

chat("Which football team is known as the Lionesses?")

# > return usage(self.input_tokens+b.input_tokens, self.output_tokens+b.output_tokens, getattr(self,'cache_creation_input_tokens',0)+getattr(b,'cache_creation_input_tokens',0), getattr(self,'cache_read_input_tokens',0)+getattr(b,'cache_read_input_tokens',0))
# >
# > TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
```
Here's the requirements.txt. I install these packages in a new environment.
```
fastcore==1.7.28
anthropic==0.42.0
toolslm==0.1.0
msglm==0.0.4
nbdev==2.3.34
boto3==1.35.92
claudette==0.1.1
ipykernel==6.29.5
```
(Unless I'm doing something wrong) the issue seems to be with getting the `cache_creation_input_tokens` and `cache_read_input_tokens` attributes: these are actually None rather than missing. So `getattr(b,'cache_creation_input_tokens',0)` returns None rather than 0.

Something like the following fixes the issue. (It's verbose though and there's probably a better solution; `__repr__` and `total` need fixing as well.)

``` python
@patch
def __add__(self: Usage, b: Usage):
"Add together each of `input_tokens` and `output_tokens`"
cache_creation_input_tokens = getattr(self, "cache_creation_input_tokens", 0) or 0
cache_creation_input_tokens += getattr(b, "cache_creation_input_tokens", 0) or 0
cache_read_input_tokens = getattr(self, "cache_read_input_tokens", 0) or 0
cache_read_input_tokens += getattr(b, "cache_read_input_tokens", 0) or 0
return usage(
self.input_tokens + b.input_tokens,
self.output_tokens + b.output_tokens,
cache_creation_input_tokens,
cache_read_input_tokens,
)
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the Usage.__add__ implementation referenced in the traceback and inspect usage, __repr__, and total handling of optional cache token attributes. Reproduce the supplied AnthropicBedrock and Claude 3.5 Sonnet example, then verify aggregation works when those attributes are None, including the displayed representation and total.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.