Add Pluggable LLM Provider & Model Selection for Improved Redundancy and Cost Efficiency

📌Description

This PR introduces a pluggable LLM provider architecture that allows users to dynamically choose their LLM provider (e.g., Groq, Gemini, OpenAI, open-source models) and specific model at runtime.

The goal is to reduce dependency on a single AI provider, improve system resilience, and align with Perspective-AI’s mission of decentralization, user choice, and cost-efficient AI usage.

🎯 Motivation

Perspective-AI aims to counter algorithmic echo chambers by presenting balanced, multi-perspective analysis. Relying on a single LLM provider introduces:

Vendor lock-in

Single point of failure

Cost scalability issues

Reduced transparency and flexibility

By enabling multiple LLM backends, we:

Improve system redundancy

Encourage open-source LLM adoption

Allow users to select models based on cost, latency, or quality

Make the system more future-proof and extensible

✨ Key Features Added
1. LLM Provider Selection

Users can now choose from multiple LLM providers, such as:

Groq

Gemini

OpenAI

Open-source / self-hosted LLMs (future-ready)

This selection can be configured via:

API request parameters

UI-level controls (frontend-ready)

Environment-level defaults

2. Model-Level Granularity

Each provider exposes its available models, allowing users to select:

Faster vs. more capable models

Cost-efficient open-source alternatives

Task-specific models for reasoning or summarization

Example:

Groq → llama3-70b, mixtral

Gemini → gemini-pro

Open-source → Mistral, LLaMA, etc.

3. Unified LLM Interface

A provider-agnostic abstraction layer ensures:

Clean separation between business logic and LLM implementations

Easy onboarding of new providers

Minimal refactoring when switching models

This keeps LangChain/LangGraph workflows intact while enabling backend flexibility.

4. Cost-Efficient & Open-Source Friendly

Default preference can be given to open-source or low-cost LLMs

Reduces reliance on expensive proprietary APIs

Supports the long-term sustainability of the project

🧠 Architectural Impact

Before:
Single LLM provider → tightly coupled → limited scalability

After:
Pluggable LLM layer → provider abstraction → dynamic model routing

This aligns with Perspective-AI’s broader architecture of:

Modular design

Redundancy in critical dependencies

Transparent AI decision pipelines

🔐 Security & Configuration

API keys are managed via environment variables

No provider-specific secrets are exposed to the frontend

Safe fallbacks ensure graceful degradation if a provider fails

📈 Expected Outcomes

Reduced vendor lock-in

Improved system resilience

Lower operational costs

Better experimentation with reasoning quality

Stronger alignment with open-source principles

🛠️ Future Scope

Auto-routing models based on task complexity

User-defined cost/latency preferences

Self-hosted local LLM support

Provider benchmarking & quality comparison UI

@ParagGhatage Assign this to me !!