Feature Proposal: Multi-LLM Support for Redundancy, Cost Optimization, and User Selection

Description

Currently, DebateAI relies on a single LLM provider for AI-powered debates. This creates potential risks related to availability, rate limits, token constraints, and cost variability. A single-provider dependency may also limit future extensibility and experimentation.

Solution

I propose introducing a provider-agnostic Multi-LLM architecture that allows DebateAI to integrate and switch between multiple Large Language Model (LLM) providers.

Key aspects of the proposed solution:

Abstract the LLM logic behind a common interface

Support multiple providers (e.g., Gemini, OpenAI, Claude, open-source LLMs)

Allow runtime selection of the LLM per debate room

Enable fallback mechanisms if a provider becomes unavailable

Why this is useful

Redundancy & Reliability: Ensures AI debate features remain available even if one provider fails or is rate-limited.

Cost Efficiency: Different models can be selected based on cost-per-token, reducing operational expenses.

Scalability: Enables usage of models with larger context windows for long-form debates.

Vendor Neutrality: Avoids lock-in and aligns with open-source sustainability principles.

Proposed High-Level Design

Introduce an LLM service layer/interface in the backend

Implement provider-specific adapters (e.g., GeminiAdapter, OpenAIAdapter)

Select provider based on configuration or user preference

Maintain graceful degradation to human-only debates if no LLM is configured

User Impact

Users can choose an LLM when creating or joining a debate room

Flexibility to switch between faster/cheaper models and more powerful models

No disruption to existing human-vs-human debate workflows

Additional Context

This feature aligns well with GSoC-style contributions, as it involves:

Backend architecture design

Clean abstractions

Extensibility for future contributors

Real-world scalability and cost considerations

I would be happy to work on this feature and discuss the design further with maintainers.

@Zahnentferner @keshav-nischal Assign this to me!!