AgentOps-AI / AgentOps-AI/tokencost

Length specific pricing bands for gemini-1.5-flash-latest

Aberta
#53 5 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
2k
Forks
105
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

I saw on https://ai.google.dev/pricing for the latest Gemini models they have 2 different bands of pricing rules, based on the length
To support this, the pricing logic and data structure might need some changes.

Maybe something like this:
```python
def calculate_cost_by_tokens(model_name, input_tokens, output_tokens):
prices = model_prices[model_name]

# Check for the presence of specific token limits and pricing structures
if 'input_cost_per_token_short' in prices and 'max_input_tokens_short' in prices:
max_input_tokens_short = prices['max_input_tokens_short']
if input_tokens <= max_input_tokens_short:
input_cost_per_token = prices['input_cost_per_token_short']
output_cost_per_token = prices['output_cost_per_token_short']
else:
input_cost_per_token = prices['input_cost_per_token_long']
output_cost_per_token = prices['output_cost_per_token_long']
else:
input_cost_per_token = prices['input_cost_per_token']
output_cost_per_token = prices['output_cost_per_token']

input_cost = input_tokens * input_cost_per_token
output_cost = output_tokens * output_cost_per_token
total_cost = input_cost + output_cost

return total_cost
```
So the pricing data might need to be stored like this:
```json
{
"gemini-1.5-flash-latest": {
"max_tokens": 8192,
"max_input_tokens": 1000000,
"max_input_tokens_short": 128000,
"max_output_tokens": 8192,
"input_cost_per_token_short": 3.5e-07,
"input_cost_per_token_long": 7e-07,
"output_cost_per_token_short": 1.0500000000000001e-06,
"output_cost_per_token_long": 2.1000000000000002e-06,
"litellm_provider": "vertex_ai-language-models",
"mode": "chat",
"supports_function_calling": true,
"supports_vision": true,
"source": "https://ai.google.dev/pricing"
}
}
```


**Proposed new optional properties:**

- `max_input_tokens_short`: The maximum number of input tokens for the lower pricing band.
- `input_cost_per_token_short`: Cost per input token when the input token count is within the `max_input_tokens_short` limit.
- `input_cost_per_token_long`: Cost per input token when the input token count exceeds the `max_input_tokens_short` limit.
- `output_cost_per_token_short`: Cost per output token when the input token count is within the `max_input_tokens_short` limit.
- `output_cost_per_token_long`: Cost per output token when the input token count exceeds the `max_input_tokens_short` limit.

Obviously it'd need tests and the numbers reviewing. Wouldn't be surprised if I've got at least 0 off-by-one errors! ;)

Seems like a very useful library, I didn't find the info I needed so hope this helps.
What do you think?
@areibman

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.