AgentOps-AI / AgentOps-AI/tokencost
Length specific pricing bands for gemini-1.5-flash-latest
- Linguagem predominante
- Python
- Estrelas
- 2k
- Forks
- 105
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
I saw on https://ai.google.dev/pricing for the latest Gemini models they have 2 different bands of pricing rules, based on the length
To support this, the pricing logic and data structure might need some changes.
Maybe something like this:
```python
def calculate_cost_by_tokens(model_name, input_tokens, output_tokens):
prices = model_prices[model_name]
# Check for the presence of specific token limits and pricing structures
if 'input_cost_per_token_short' in prices and 'max_input_tokens_short' in prices:
max_input_tokens_short = prices['max_input_tokens_short']
if input_tokens <= max_input_tokens_short:
input_cost_per_token = prices['input_cost_per_token_short']
output_cost_per_token = prices['output_cost_per_token_short']
else:
input_cost_per_token = prices['input_cost_per_token_long']
output_cost_per_token = prices['output_cost_per_token_long']
else:
input_cost_per_token = prices['input_cost_per_token']
output_cost_per_token = prices['output_cost_per_token']
input_cost = input_tokens * input_cost_per_token
output_cost = output_tokens * output_cost_per_token
total_cost = input_cost + output_cost
return total_cost
```
So the pricing data might need to be stored like this:
```json
{
"gemini-1.5-flash-latest": {
"max_tokens": 8192,
"max_input_tokens": 1000000,
"max_input_tokens_short": 128000,
"max_output_tokens": 8192,
"input_cost_per_token_short": 3.5e-07,
"input_cost_per_token_long": 7e-07,
"output_cost_per_token_short": 1.0500000000000001e-06,
"output_cost_per_token_long": 2.1000000000000002e-06,
"litellm_provider": "vertex_ai-language-models",
"mode": "chat",
"supports_function_calling": true,
"supports_vision": true,
"source": "https://ai.google.dev/pricing"
}
}
```
**Proposed new optional properties:**
- `max_input_tokens_short`: The maximum number of input tokens for the lower pricing band.
- `input_cost_per_token_short`: Cost per input token when the input token count is within the `max_input_tokens_short` limit.
- `input_cost_per_token_long`: Cost per input token when the input token count exceeds the `max_input_tokens_short` limit.
- `output_cost_per_token_short`: Cost per output token when the input token count is within the `max_input_tokens_short` limit.
- `output_cost_per_token_long`: Cost per output token when the input token count exceeds the `max_input_tokens_short` limit.
Obviously it'd need tests and the numbers reviewing. Wouldn't be surprised if I've got at least 0 off-by-one errors! ;)
Seems like a very useful library, I didn't find the info I needed so hope this helps.
What do you think?
@areibman
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Avaliação
Esta issue ainda não foi avaliada.