Azure / Azure/GPT-RAG

[Feature request] optional cost & deployment dashboard provisioned with GPT-RAG (production vs Zero-Trust scaffolding breakdown)

Open
#694 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
321
Avg merge
6h 22m
Merged PRs (30d)
26

Description

## Summary
As a customer running a GPT-RAG POC/MVP, I'm always frustrated when I look at the Azure bill and can't tell what the solution will actually cost in production. A large part of the cost comes from **Zero-Trust scaffolding** (a jump box VM for access, private endpoints, elevated tiers) that is only there for the secure POC and won't exist in production, but there is no easy, shareable view that makes that distinction. This requests an **optional dashboard, provisioned at deploy time (behind a flag / add-on)**, that shows the deployed solution's cost **service by service** and clearly separates **core/production** resources from **POC-only scaffolding**.

## Why will we implement this?
- **Problem / opportunity:** `Cost is a near-universal customer question in POCs/MVPs ("how much will this cost, and how does it scale at 10 vs 100 users?"). The data exists in Azure Cost Analysis but is not consumable or shareable, and it does not tell the story of core vs scaffolding. The jump box VM is often the single biggest line item yet is not part of the production workload, so customers overestimate production cost. Today this is explained manually, per engagement.`
- **Business value / outcome:** `Faster cost clarity and trust, fewer repeated manual explanations, smoother POC-to-production conversations, and higher accelerator adoption.`
- **Success metrics (how we know it worked):** `Fewer cost-related objections/questions per engagement; customers can self-serve the "POC vs production cost" answer; reduction in manually built cost decks; adoption of the flag across deployments.`

## What does it do? (Functional Overview)
- **Core behavior:** `An optional deploy-time flag provisions a live cost dashboard scoped to the deployment's resource group. It shows per-service cost and tags each resource as core/production or Zero-Trust scaffolding (jump box VM, private endpoints, elevated tiers), with a short note on why each is deployed. Optionally surfaces usage metrics (model tokens, orchestrator-to-LLM requests) and budget/threshold alerts.`
- **Data collection / storage needs:** `Reads Azure Cost Management / Cost Analysis data for the resource group and Azure Monitor metrics for token/request usage. Applies resource tags (core vs scaffolding) at deploy time. No new customer data store required.`
- **Data analysis / reporting needs:** `Per-service cost breakdown; core-vs-scaffolding split; a production-cost view (excluding POC-only resources); a shareable/exportable dashboard (e.g., Azure Workbook) understandable by a non-technical stakeholder.`
- **Nice to have (stretch goals):** `Budget/threshold alerts; cost projection at N users; export/share for finance stakeholders; extend to the full solution (VM, database, endpoints) beyond the agent itself.`

## Components
- **Components (check all that apply):**
- [x] gpt-rag-orchestrator `(emit usage/token/request metrics for the usage view)`
- [ ] gpt-rag-ingestion
- [x] gpt-rag-ui `(host the dashboard / admin page)`
- [ ] gpt-rag-mcp

_Note: the core of this feature is IaC/Bicep provisioning of the dashboard and resource tagging, which may sit in the infra/deployment layer outside the four components above._

Contributor guide

Open the contributing guide

Research direction

Start by locating the IaC/Bicep deployment layer and the gpt-rag-orchestrator and gpt-rag-ui entry points. Confirm how an optional deploy-time flag, resource tags, Azure Cost Management data, and Azure Monitor metrics would fit together. Done means a scoped dashboard can show service costs and separate core production resources from Zero-Trust scaffolding.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
cloud, infrastructure, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.