AgentOps-AI / AgentOps-AI/tokencost

Length specific pricing bands for gemini-1.5-flash-latest

未关闭
#53 5 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
2k
派生
105
PR 合并指标
30 天内没有已合并 PR

描述

I saw on https://ai.google.dev/pricing for the latest Gemini models they have 2 different bands of pricing rules, based on the length
To support this, the pricing logic and data structure might need some changes.

Maybe something like this:
```python
def calculate_cost_by_tokens(model_name, input_tokens, output_tokens):
prices = model_prices[model_name]

# Check for the presence of specific token limits and pricing structures
if 'input_cost_per_token_short' in prices and 'max_input_tokens_short' in prices:
max_input_tokens_short = prices['max_input_tokens_short']
if input_tokens <= max_input_tokens_short:
input_cost_per_token = prices['input_cost_per_token_short']
output_cost_per_token = prices['output_cost_per_token_short']
else:
input_cost_per_token = prices['input_cost_per_token_long']
output_cost_per_token = prices['output_cost_per_token_long']
else:
input_cost_per_token = prices['input_cost_per_token']
output_cost_per_token = prices['output_cost_per_token']

input_cost = input_tokens * input_cost_per_token
output_cost = output_tokens * output_cost_per_token
total_cost = input_cost + output_cost

return total_cost
```
So the pricing data might need to be stored like this:
```json
{
"gemini-1.5-flash-latest": {
"max_tokens": 8192,
"max_input_tokens": 1000000,
"max_input_tokens_short": 128000,
"max_output_tokens": 8192,
"input_cost_per_token_short": 3.5e-07,
"input_cost_per_token_long": 7e-07,
"output_cost_per_token_short": 1.0500000000000001e-06,
"output_cost_per_token_long": 2.1000000000000002e-06,
"litellm_provider": "vertex_ai-language-models",
"mode": "chat",
"supports_function_calling": true,
"supports_vision": true,
"source": "https://ai.google.dev/pricing"
}
}
```


**Proposed new optional properties:**

- `max_input_tokens_short`: The maximum number of input tokens for the lower pricing band.
- `input_cost_per_token_short`: Cost per input token when the input token count is within the `max_input_tokens_short` limit.
- `input_cost_per_token_long`: Cost per input token when the input token count exceeds the `max_input_tokens_short` limit.
- `output_cost_per_token_short`: Cost per output token when the input token count is within the `max_input_tokens_short` limit.
- `output_cost_per_token_long`: Cost per output token when the input token count exceeds the `max_input_tokens_short` limit.

Obviously it'd need tests and the numbers reviewing. Wouldn't be surprised if I've got at least 0 off-by-one errors! ;)

Seems like a very useful library, I didn't find the info I needed so hope this helps.
What do you think?
@areibman

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。