Azure-Samples / Azure-Samples/AI-Gateway
Cost control and monitoring per user
- Dominant language
- Jupyter Notebook
- Stars
- 986
- Forks
- 513
- Avg merge
- 12m
- Merged PRs (30d)
- 1
Description
There is no out-of-the-box functionality to set budgets in $$$ per user and also let the user have a observability for their own usage / quota.
Setting budget per user can solve the noise neighbor problem where one user can consume the whole budget in one day.
Each API (like completions, responses, messages, image, audio, video) emits different information accoring to API specification.
Each model (like GPT-4, GPT4o, GPT5.5, GPT5.6-LUNA, GPT5.6-TERRA, GPT5.6-SOL, CLAUDE-OPUS-5) charges in different rates.
Each provider has each own cost policies, like input, input-cache-5m, input-cache-1h, output, output-reasoning, etc...
APIM does not support all authentication headers and schemas to support all AI frameworks/tools.
For this reason I have created this repo that fullfils all the above.
https://github.com/koureasstavros/AzureAIGatewayCostMechanism
It would be great to see all these features out of the box into the Azure AI Gateway.
Contributor guide
Research direction
No project files, tests, or entry points are named. Start by reviewing the linked AzureAIGatewayCostMechanism repository and the Azure AI Gateway implementation to define how per-user budgets, usage visibility, provider pricing, and authentication support should fit. Done means an agreed, tested design and implementation covering the requested APIs, models, providers, and quotas.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- ai, api, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100