diegosouzapw / diegosouzapw/OmniRoute
fix(providers): ModelSync imports batch-only variants (*:batch) into chat routing — 91 Batch API rejections
- Dominant language
- TypeScript
- Stars
- 66.7k
- Forks
- 9.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 512
Description
**OmniRoute Version:** 3.8.50
**Installation Method:** npm (global)
**Operating System:** Linux
**OS Version:** Ubuntu / Linux 6.17.0-1020-oracle (arm64)
**Node.js Version:** v24.16.0
**Provider(s) Involved:** OpenRouter (synced `*:batch` variants), GitHub Copilot (`github`)
**Model(s) Involved:** `claude-*:batch`, `gemini-*:batch`, `minimax-m3:batch`, `inkling:batch` — 77 synced variants; plus unsupported Copilot models
**Client Tool:** Any client routed through OmniRoute
## Description
ModelSync imports upstream model variants that can never serve a normal chat request:
- **77 `*:batch` routes** were synced from OpenRouter into the local model→provider map (`key_value`), e.g. `gemini-3.6-flash:batch` → `openrouter/google/gemini-3.6-flash:batch`. When a client requests the base model and the router lands on a `:batch` variant, the upstream rejects with `404 This model is only available through the Batch API` — **91 errors** in ~41 h, the #3 error class. Provider-side warnings confirm the loop: `[provider] Node model not found (N) for google/gemini-*:batch - locking mode`.
- Related catalog staleness on GitHub Copilot: `400 The requested model is not supported.` — **34 errors** in the same window, i.e. synced models the upstream no longer serves.
## Steps to Reproduce
1. Let ModelSync sync the catalog (observed `ModelSync:9router synced 387 model(s)`).
2. Send a normal chat request for e.g. `gemini-3.6-flash` (or let a combo/failover pick it).
3. Router selects a `*:batch` variant → upstream 404 `only available through the Batch API`.
## Expected Behavior
- `:batch`, embedding-only, moderation, and other non-chat variants are excluded from chat routing (or tagged batch-only and only used for `/api/beta/batches` requests).
- Models rejected as "not supported" are quarantined/removed after N consecutive rejections.
## Actual Behavior
Batch variants and stale models stay in the routing table, producing recurring 404/400 rejections and wasted failover attempts.
## Test Impact
Needs a new unit test — ModelSync filter rejects `*:batch` variants; plus integration test asserting synced catalog contains no batch-only routes.
## Error Logs / Output
```shell
[ERROR] [404]: This model is only available through the Batch API. Use the /api/beta/batches endpoint instead. # ×91
[ERROR] [400]: The requested model is not supported. # ×34 (github)
[provider] Node model not found (N) for google/gemini-3.5-flash:batch - locking mode # warn ×80+
```
## Possible Solutions
- Filter `:batch` / non-chat suffixes and capability flags during ModelSync import.
- Auto-quarantine: mark a model unavailable after repeated `not supported`/`Batch API` rejections.
- Surface "routable vs catalog" model counts in the dashboard.
## Validation Plan
- After sync, `key_value` contains no `*:batch` chat routes.
- A chat request never resolves to a `:batch` upstream.
## Additional Context
Local mitigation applied: deleted the 77 `*:batch` route keys from `key_value` — they will presumably return on the next ModelSync cycle until this is fixed upstream.
Contributor guide
Research direction
Start by locating ModelSync's catalog import and the key_value model-to-provider routing map, then inspect how OpenRouter and GitHub Copilot models are represented. Add unit coverage for excluding :batch variants and an integration check that synced chat routes contain none; validation is that chat requests never resolve to a batch-only upstream.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend-api-design
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 56/100