wso2 / wso2/api-platform

[Feature]: Multi-provider LLM proxies and selectable inbound interface as an experience in AI Workspace

Open
#3,368 2 comments 0 reactions 1 assignee View on GitHub

@veejask-41 is already working on this.

Since Sep 4, 2026.

Area/AIGateway Area/AIPolicies Area/AIWorkspace Aspect/AI Aspect/API Aspect/UI Aspect/UX Design Done Priority/High Severity/Major Type/New Feature
Dominant language
Go
Stars
71
Forks
111
Avg merge
1d 14h
Merged PRs (30d)
110

Description

Please select the area the issue is related to

Gateway, Platform API, AI Workspace

Please select the aspect the issue is related to

Aspect/API (API backends, definitions, contracts, interfaces, OpenAPI), Aspect/UI (Frontend layouts, components, styling), Aspect/UX (User experience, flows, usability, clarity), Aspect/AI (AI/LLM integration, MCP, AI readiness)

Suggested Feature
Problem

The gateway supports routing one LLM proxy to multiple providers, with per-provider transformer policies converting between wire formats. None of this is reachable from AI Workspace.

Two gaps today:

  1. A proxy connects to exactly one provider. additionalProviders cannot be expressed in the UI.
  2. The inbound interface is silently inherited. A proxy's request format is copied from its primary provider at creation and never presented as a choice, so "expose an OpenAI-compatible endpoint in front of a Mistral provider" is not expressible.

The underlying capability already exists end to end in the gateway (additionalProviders, llm-header-router, five openai-to-* transformer policies) and is partially wired through Platform API.

Proposed solution

Introduce inbound interface as an explicit proxy setting, and allow attaching multiple providers to a proxy.

Core model

The inbound interface is the request format client applications send. It defaults to the primary provider's template and is identified by a template handle (openai, anthropic, gemini, mistralai, …). Its only job is resolving which transformer each attached provider needs:

Condition Result
provider template == interface No transformer needed
provider template != interface Requires policy {interface}-to-{template}-transformer
that policy is absent Provider can be attached but proxy cannot be deployed without required transformer

This one rule covers both gaps. Multi-provider fan-out is many providers reconciled to one interface; "OpenAI interface on a Mistral proxy" is one provider whose template differs from the interface.

Related Issues

No response

Steps to Verify
  • Design Document — A detailed design document has been created and reviewed, covering architecture, data flow, and edge cases.
  • Design Mail — A design summary email has been sent to relevant stakeholders for awareness and feedback.
  • Code Review — All code changes have been peer-reviewed and approved according to the project's review standards.
  • Testing Complete — Adequate unit, integration, and/or end-to-end tests have been written and are passing.
  • Documentation Review — User-facing and/or developer documentation has been updated to reflect the new feature and reviewed.
  • Feature Complete — The feature is fully implemented, all checklist items above are done, and it is ready for release.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.