Project : Reasoning Agents (Azure AI Foundry) - AI-Driven Service Intelligence: Multi-Agent System for Automotive Service Operations

Track

Reasoning Agents (Azure AI Foundry)

Project Name

AI-Driven Service Intelligence: Multi-Agent System for Automotive Service Operations

GitHub Username

@Shrusti13, @dips365, @jitenparmar, @manishmlv50, @bhushang19

Repository URL

https://github.com/manishmlv50/AI-Powered-Service-Intelligene

Project Description

Service Advisor AI Intelligence is an AI-powered multi-agent system that augments existing workshop systems (not replaces them), transforming raw complaints and OBD codes into structured job cards, intelligent estimates, proactive communication, and live operational visibility.

Built on Azure OpenAI + FastAPI + React + Azure SQL, it is designed to be enterprise-ready, synthetic data, and demo-friendly.

Demo Video or Screenshots

Demo Video : https://github.com/manishmlv50/AI-Powered-Service-Intelligene/tree/main/docs/assets/demo-video
Screenshots : https://github.com/manishmlv50/AI-Powered-Service-Intelligene/tree/main/docs/assets/screenshots

Primary Programming Language

Python

Key Technologies Used

Frontend

• React + Vite

Backend

• FastAPI

AI & Intelligence

• Azure OpenAI (Responses API)
• Azure OpenAI (Reasoning model)
• Azure AI Speech Service

Data Layer

• Azure SQL (Primary)
• JSON synthetic fallback

Submission Type

Team (2-4 members)

Team Members

• @bhushang19 - Product Owner and Microsoft MVP
• @dips365 - Backend and AI Engineer
• @jitenparmar - Backend and AI Engineer
• @manishmlv50 - Frontend and AI Engineer
• @Shrusti13 - Lead AI Engineer and Microsoft MVP

Submission Requirements

• My project meets the track-specific challenge requirements
• My repository includes a comprehensive README.md with setup instructions
• My code does not contain hardcoded API keys or secrets
• I have included demo materials (video or screenshots)
• My project is my own work with proper attribution for any third-party code
• I agree to the Code of Conduct
• I have read and agree to the Disclaimer
• My submission does NOT contain any confidential, proprietary, or sensitive information
• I confirm I have the rights to submit this content and grant the necessary licenses

Quick Setup Summary

Setup Summary

Prerequisites

• Python 3.11+, Node.js 18+, npm
• Azure OpenAI resource with Responses API deployment
• Azure SQL Database (optional - JSON fallback available)

Setup

Backend

cd sourcecode
python -m venv .venv
.venv\Scripts\activate  # Windows
# source .venv/bin/activate  # macOS/Linux
pip install -r requirements.txt

Frontend

cd sourcecode/frontend
npm install

Configuration

cp .env.example sourcecode/.env

Edit sourcecode/.env with your Azure credentials (see .env.example for required variables).

Run

Backend:

cd sourcecode
python -m uvicorn app.main:app --reload --host 127.0.0.1 --port 8000

→ http://127.0.0.1:8000/docs

Frontend:

cd sourcecode/frontend
npm run dev

→ http://localhost:5173

Quick Test

curl -X POST http://127.0.0.1:8000/api/agents/master \
  -H "Content-Type: application/json" \
  -d '{"user_input": "brake noise when stopping"}'

Technical Highlights

Implementation Highlights

What We're Most Proud Of

Deterministic Multi-Agent Orchestration

Single-dispatch router pattern where Master Agent routes to exactly one specialist agent per request. Zero computation overhead pure routing with explicit, deterministic rules. Master Agent never generates responses, only routes and returns tool outputs.

Schema-Bound LLM Outputs

Dual-layer validation: Azure OpenAI Responses API output_schema + runtime Pydantic validation. Ensures type safety and prevents malformed JSON. 99%+ schema compliance rate.

Tool-First Agent Design

Mandatory tool-calling workflow agents must query SQL database before generating responses. Prevents hallucination by ensuring all vehicle data, fault codes, and parts come from authoritative sources.

Real-Time Speech-to-Text

WebSocket-based streaming transcription processing PCM audio chunks (16kHz, 16-bit, mono) in real-time. Provides partial transcriptions as user speaks.

OBD Code Intelligence

Domain-aware system mapping raw OBD fault codes to structured job cards with repair tasks, parts recommendations, and service type classification (urgent_repair, repair, diagnostic, maintenance).

---

Key Technical Decisions

Single-Dispatch Router: Routes to one agent per request (never chains). Prioritizes clarity and performance over flexibility.

Prompt Engineering Over Fine-Tuning: Faster iteration, easier modification, works with any Azure OpenAI deployment. No model retraining required.

Repository Abstraction: Unified db_service.py interface abstracts SQL vs. JSON.

Structured Output at Agent Level: Pydantic schemas passed directly to Azure OpenAI Responses API. Eliminates manual JSON parsing.

Tool-Calling as Data Authority: Mandatory SQL lookups prevent LLM hallucination. Job cards and estimates always based on real data.

---

Architecture Strengths

• Clear separation of concerns (agents, API, data layer, frontend)
• End-to-end Pydantic validation
• Resilient fallbacks for unavailable Azure services
• Stateless agents enable horizontal scaling

---

Unique Differentiators

1. Domain-Specific Intelligence: Agents understand automotive workflows, OBD codes, and parts/labor estimation
2. Production-Ready Patterns: Enterprise patterns (repository abstraction, schema validation)
3. Multi-Modal Input: Text, voice, and file uploads in unified workflow
4. Zero-Config Fallbacks: Automatic data source switching
5. Deterministic Routing: Explicit, debuggable agent selection vs. black-box LLM routing

Challenges & Learnings

Challenges and Learnings

Challenges Faced

Multi-Agent Orchestration

Coordinating multiple specialized agents (Intake, Estimation, Communication) through a master orchestrator required careful state management and context passing. Ensuring each agent received the right context while maintaining conversation flow was complex.

Structured LLM Outputs

Getting consistent, schema-validated JSON responses from Azure OpenAI required extensive prompt engineering and Pydantic validation. Balancing flexibility with structure was key to reliable parsing.

OBD File Processing

Converting unstructured OBD diagnostic files into actionable fault codes and system mappings required domain knowledge integration and robust parsing logic.

Real-Time UI Synchronization

Keeping the React frontend in sync with backend agent states and multi-step workflows required careful state management and API design.

Azure Services Integration

Configuring Azure OpenAI endpoints, managing API keys securely, and handling rate limits while maintaining graceful fallbacks to synthetic data.

Key Learnings

Agent Framework Architecture

Learned to design agent systems with clear separation of concerns—each agent handles a specific domain while the orchestrator manages routing and context flow.

Prompt Engineering for Production

Discovered the importance of structured prompts with explicit output schemas, few-shot examples, and validation layers for reliable AI responses in operational systems.

Domain Knowledge Integration

Understanding automotive service workflows, OBD codes, and parts/labor estimation helped create more accurate and useful AI responses.

---

Contact Information

shrustishah1395@gmail.com

Country/Region

India

This issue is a project submission rather than a scoped change request. Review the linked AI-Powered-Service-Intelligene repository, its sourcecode backend and frontend, and the demo materials to understand the existing system. No specific file, test, requested change, or completion criteria are provided.