[Feature request] native inline chart / data visualization rendering in GPT-RAG responses
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 321
- Avg merge
- 6h 22m
- Merged PRs (30d)
- 26
Description
## Summary
The text-first GPT-RAG flow returns text and tables, but I'm always frustrated when the answer needs a **chart to be useful** and there is no native way to render one inline. This has come up across more than one customer engagement, where an inline chart is a primary way of consuming the answer, not an optional add-on. This requests **native support for returning data visualizations (charts/graphs) inline in the response**, in addition to text and tables. A **simple first version** (a few basic chart types) would already deliver value.
## Why will we implement this?
- **Problem / opportunity:** `Several use cases need the grounded answer to include a rendered chart. Users are accustomed to BI-dashboard-style visuals (e.g., Power BI) and expect the in-chat chart to feel familiar, just delivered inline instead of a separate dashboard. Today this requires custom, per-project front-end work (an app/bot that generates chart images) that is not part of the accelerator, does not render consistently across surfaces (web app, Teams, Copilot), and is not multi-user or governed by default.`
- **Business value / outcome:** `Closes a recurring capability gap, removes duplicated per-project effort, delivers faster and more accessible answers (a governed, cited answer with a chart in seconds), and increases accelerator adoption.`
- **Success metrics (how we know it worked):** `A user can ask a question and get an inline chart with no bespoke per-project front-end code; adoption across engagements; fewer custom chart apps built per engagement; charts render acceptably in Teams and Copilot, multi-user and governed.`
## What does it do? (Functional Overview)
- **Core behavior:** `Generate and render charts inline in the response from the data the agent has already grounded/retrieved, with consistent rendering across the surfaces GPT-RAG targets (web app, Teams, Copilot) and correct sizing. Chart type is auto-selected from the data/intent or explicitly requestable. Charts carry the same grounding, citation, and traceability as the text answer, at a visual quality comparable to standard BI visuals (labels, legends, axes).`
- **Data collection / storage needs:** `Uses the query results / grounded data the agent already retrieves. No new customer data store required. May store a small chart specification/config alongside the answer.`
- **Data analysis / reporting needs:** `Render a set of chart types. Phase 1 (basic): line / time-series trend, bar and grouped bar, formatted tables. Later: stacked bar, pie / donut (share / composition), histogram / distribution, scatter / correlation, series with a reference or threshold overlay.`
- **Nice to have (stretch goals):** `Automatic chart-type selection from intent; richer chart types above; exportable/shareable chart images; sanitized sample mockups attached to guide implementation (initial expectation is simple BI-style visuals, e.g., a metric over time or a comparison across a few categories).`
## Components
- **Components (check all that apply):**
- [x] gpt-rag-orchestrator `(produce the chart / chart spec from grounded results)`
- [ ] gpt-rag-ingestion
- [x] gpt-rag-ui `(render the chart inline in the response)`
- [ ] gpt-rag-mcp
_Context: demand observed across more than one customer engagement. Current workaround is a custom app/bot that returns PNG charts, which is single-user by default, renders inconsistently or too small in some surfaces (e.g., Copilot), is not IaC / not part of the accelerator / not governed to standard, and duplicates effort per engagement._
Contributor guide
Research direction
Start by tracing the gpt-rag-orchestrator flow from grounded query results to the response, then inspect gpt-rag-ui for rendering across the supported surfaces. Define a small Phase 1 scope around line/time-series, bar or grouped-bar charts, and formatted tables. Done means the chart is inline, correctly sized, cited and traceable, and renders acceptably in the web app, Teams and Copilot.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, python
- Domain
- ai, data-visualization, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100