router-for-me / router-for-me/CLIProxyAPI

[Feature] Add per-service protocol selection for custom providers

Open
#5,376 2 comments 0 reactions 1 assignee View on GitHub

@hkfires is already working on this.

Since Sep 3, 2026.

pending
Dominant language
Go
Stars
52.5k
Forks
7.9k
Avg merge
1d 3h
Merged PRs (30d)
60

Description

Background

Many third-party AI providers expose more than one native API protocol, such as:

  • OpenAI Chat Completions
  • OpenAI Responses
  • Anthropic Messages

At the moment, third-party API keys can generally only be configured through
openai-compatibility, which assumes the OpenAI Chat Completions format. When
the client protocol differs from the upstream protocol, CLIProxyAPI has to
translate the request and response.

This can lose provider-specific features such as native thinking controls,
tool-call details, image handling, usage fields, and protocol-specific
parameters.

This is a follow-up to #4756, which was converted to a discussion. The proposal
below turns that request into a concrete configuration model.

Proposed configuration

Add a separate custom-provider configuration family. Each entry represents
one upstream service and selects exactly one protocol.

custom-provider:
  - name: deepseek-completions
    protocol: completions
    base-url: https://api.deepseek.com/v1
    api-key-entries:
      - api-key: ${DEEPSEEK_API_KEY}
    models:
      - name: deepseek-chat
        alias: deepseek-chat

  - name: openrouter-responses
    protocol: responses
    base-url: https://openrouter.ai/api/v1
    api-key-entries:
      - api-key: ${OPENROUTER_API_KEY}
    models:
      - name: openai/gpt-4o
        alias: openrouter-gpt-4o

  - name: provider-messages
    protocol: messages
    base-url: https://api.example.com/v1
    api-key-entries:
      - api-key: ${PROVIDER_API_KEY}
    models:
      - name: example-model
        alias: example-model

The existing openai-compatibility configuration should remain unchanged.

Protocol behavior

Protocol Upstream endpoint Authentication
completions <base-url>/chat/completions Authorization: Bearer <key>
responses <base-url>/responses Authorization: Bearer <key>
messages <base-url>/messages x-api-key: <key>

For Anthropic Messages, the implementation should add a default
anthropic-version: 2023-06-01 header while still allowing custom headers.

The configured base-url should be the API root, including any version prefix,
for example https://api.example.com/v1.

Expected behavior

  • Each custom provider entry uses one protocol.
  • The same service can be configured multiple times with different names when
    different protocols are required.
  • Entries with the same normalized name should be rejected.
  • Empty names, empty base URLs, and unknown protocols should be rejected.
  • If protocol is omitted, it should default to completions.
  • Existing model aliases, weights, proxy settings, retries, cooldowns, image
    options, thinking options, and usage accounting should continue to work.
  • All API keys belonging to the same service should use the same executor, with
    the executor bound to that service protocol.
  • Provider selection, model registration, hot reload, and request-scoped errors
    should treat custom providers as a separate provider family.

Management API

Please expose CRUD operations for custom providers:

  • GET /v0/management/custom-provider
  • PUT /v0/management/custom-provider
  • PATCH /v0/management/custom-provider
  • DELETE /v0/management/custom-provider

The API should support the existing array and {items: [...]} request forms,
name/index based updates and deletion, protocol validation, weight validation,
and auth-index reporting.

Acceptance criteria

  • A client can call /v1/chat/completions, /v1/responses, or /v1/messages
    and route to a matching custom provider without requiring the provider to be
    represented as openai-compatibility.
  • Native protocol requests are forwarded without unnecessary translation when
    the client and upstream protocols match.
  • Cross-protocol translation remains available when the protocols differ.
  • Existing openai-compatibility configurations behave exactly as before.
  • Configuration, management API, reload, routing, proxy, retry, cooldown,
    streaming, usage, and token-counting tests cover the new provider family.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.