diegosouzapw / diegosouzapw/OmniRoute

fix(providers): ModelSync imports batch-only variants (*:batch) into chat routing — 91 Batch API rejections

Closed
#13,596 3 comments 0 reactions 0 assignees View on GitHub
acknowledged bug providers
Dominant language
TypeScript
Stars
66.7k
Forks
9.4k
Avg merge
2d 15h
Merged PRs (30d)
512

Description

**OmniRoute Version:** 3.8.50
**Installation Method:** npm (global)
**Operating System:** Linux
**OS Version:** Ubuntu / Linux 6.17.0-1020-oracle (arm64)
**Node.js Version:** v24.16.0
**Provider(s) Involved:** OpenRouter (synced `*:batch` variants), GitHub Copilot (`github`)
**Model(s) Involved:** `claude-*:batch`, `gemini-*:batch`, `minimax-m3:batch`, `inkling:batch` — 77 synced variants; plus unsupported Copilot models
**Client Tool:** Any client routed through OmniRoute

## Description

ModelSync imports upstream model variants that can never serve a normal chat request:

- **77 `*:batch` routes** were synced from OpenRouter into the local model→provider map (`key_value`), e.g. `gemini-3.6-flash:batch` → `openrouter/google/gemini-3.6-flash:batch`. When a client requests the base model and the router lands on a `:batch` variant, the upstream rejects with `404 This model is only available through the Batch API` — **91 errors** in ~41 h, the #3 error class. Provider-side warnings confirm the loop: `[provider] Node model not found (N) for google/gemini-*:batch - locking mode`.
- Related catalog staleness on GitHub Copilot: `400 The requested model is not supported.` — **34 errors** in the same window, i.e. synced models the upstream no longer serves.

## Steps to Reproduce

1. Let ModelSync sync the catalog (observed `ModelSync:9router synced 387 model(s)`).
2. Send a normal chat request for e.g. `gemini-3.6-flash` (or let a combo/failover pick it).
3. Router selects a `*:batch` variant → upstream 404 `only available through the Batch API`.

## Expected Behavior

- `:batch`, embedding-only, moderation, and other non-chat variants are excluded from chat routing (or tagged batch-only and only used for `/api/beta/batches` requests).
- Models rejected as "not supported" are quarantined/removed after N consecutive rejections.

## Actual Behavior

Batch variants and stale models stay in the routing table, producing recurring 404/400 rejections and wasted failover attempts.

## Test Impact

Needs a new unit test — ModelSync filter rejects `*:batch` variants; plus integration test asserting synced catalog contains no batch-only routes.

## Error Logs / Output

```shell
[ERROR] [404]: This model is only available through the Batch API. Use the /api/beta/batches endpoint instead. # ×91
[ERROR] [400]: The requested model is not supported. # ×34 (github)
[provider] Node model not found (N) for google/gemini-3.5-flash:batch - locking mode # warn ×80+
```

## Possible Solutions

- Filter `:batch` / non-chat suffixes and capability flags during ModelSync import.
- Auto-quarantine: mark a model unavailable after repeated `not supported`/`Batch API` rejections.
- Surface "routable vs catalog" model counts in the dashboard.

## Validation Plan

- After sync, `key_value` contains no `*:batch` chat routes.
- A chat request never resolves to a `:batch` upstream.

## Additional Context

Local mitigation applied: deleted the 77 `*:batch` route keys from `key_value` — they will presumably return on the next ModelSync cycle until this is fixed upstream.

Contributor guide

Open the contributing guide

Research direction

Start by locating ModelSync's catalog import and the key_value model-to-provider routing map, then inspect how OpenRouter and GitHub Copilot models are represented. Add unit coverage for excluding :batch variants and an integration check that synced chat routes contain none; validation is that chat requests never resolve to a batch-only upstream.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
56/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.