Azure-Samples / Azure-Samples/AI-Gateway

Cost control and monitoring per user

Open
#392 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
986
Forks
513
Avg merge
12m
Merged PRs (30d)
1

Description

There is no out-of-the-box functionality to set budgets in $$$ per user and also let the user have a observability for their own usage / quota.
Setting budget per user can solve the noise neighbor problem where one user can consume the whole budget in one day.
Each API (like completions, responses, messages, image, audio, video) emits different information accoring to API specification.
Each model (like GPT-4, GPT4o, GPT5.5, GPT5.6-LUNA, GPT5.6-TERRA, GPT5.6-SOL, CLAUDE-OPUS-5) charges in different rates.
Each provider has each own cost policies, like input, input-cache-5m, input-cache-1h, output, output-reasoning, etc...
APIM does not support all authentication headers and schemas to support all AI frameworks/tools.

For this reason I have created this repo that fullfils all the above.
https://github.com/koureasstavros/AzureAIGatewayCostMechanism

It would be great to see all these features out of the box into the Azure AI Gateway.

Contributor guide

Open the contributing guide

Research direction

No project files, tests, or entry points are named. Start by reviewing the linked AzureAIGatewayCostMechanism repository and the Azure AI Gateway implementation to define how per-user budgets, usage visibility, provider pricing, and authentication support should fit. Done means an agreed, tested design and implementation covering the requested APIs, models, providers, and quotas.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
ai, api, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.