AgentOps-AI / AgentOps-AI/tokencost

Length specific pricing bands for gemini-1.5-flash-latest

Ouverte
#53 5 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
2k
Forks
105
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

I saw on https://ai.google.dev/pricing for the latest Gemini models they have 2 different bands of pricing rules, based on the length
To support this, the pricing logic and data structure might need some changes.

Maybe something like this:
```python
def calculate_cost_by_tokens(model_name, input_tokens, output_tokens):
prices = model_prices[model_name]

# Check for the presence of specific token limits and pricing structures
if 'input_cost_per_token_short' in prices and 'max_input_tokens_short' in prices:
max_input_tokens_short = prices['max_input_tokens_short']
if input_tokens <= max_input_tokens_short:
input_cost_per_token = prices['input_cost_per_token_short']
output_cost_per_token = prices['output_cost_per_token_short']
else:
input_cost_per_token = prices['input_cost_per_token_long']
output_cost_per_token = prices['output_cost_per_token_long']
else:
input_cost_per_token = prices['input_cost_per_token']
output_cost_per_token = prices['output_cost_per_token']

input_cost = input_tokens * input_cost_per_token
output_cost = output_tokens * output_cost_per_token
total_cost = input_cost + output_cost

return total_cost
```
So the pricing data might need to be stored like this:
```json
{
"gemini-1.5-flash-latest": {
"max_tokens": 8192,
"max_input_tokens": 1000000,
"max_input_tokens_short": 128000,
"max_output_tokens": 8192,
"input_cost_per_token_short": 3.5e-07,
"input_cost_per_token_long": 7e-07,
"output_cost_per_token_short": 1.0500000000000001e-06,
"output_cost_per_token_long": 2.1000000000000002e-06,
"litellm_provider": "vertex_ai-language-models",
"mode": "chat",
"supports_function_calling": true,
"supports_vision": true,
"source": "https://ai.google.dev/pricing"
}
}
```


**Proposed new optional properties:**

- `max_input_tokens_short`: The maximum number of input tokens for the lower pricing band.
- `input_cost_per_token_short`: Cost per input token when the input token count is within the `max_input_tokens_short` limit.
- `input_cost_per_token_long`: Cost per input token when the input token count exceeds the `max_input_tokens_short` limit.
- `output_cost_per_token_short`: Cost per output token when the input token count is within the `max_input_tokens_short` limit.
- `output_cost_per_token_long`: Cost per output token when the input token count exceeds the `max_input_tokens_short` limit.

Obviously it'd need tests and the numbers reviewing. Wouldn't be surprised if I've got at least 0 off-by-one errors! ;)

Seems like a very useful library, I didn't find the info I needed so hope this helps.
What do you think?
@areibman

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.