[Feature]: OrcaRouter provider support for openbench

Start by locating the provider layer and reviewing the existing Groq, OpenRouter, and Together AI integrations, then check the README provider table and contribution guidelines. Confirm how API-key environment variables and provider/model strings are registered. Done means OrcaRouter can be selected with an environment variable and an orcarouter/ string, with the provider documented and tested.

Feature type: New model provider

Is your feature request related to a problem?

openbench is a provider-agnostic, open-source evaluation framework for language models: one bench eval CLI runs 95+ standardized benchmarks against any of 30+ providers using model strings like groq/openai/gpt-oss-120b, and it also runs private evals straight from a local path. For people comparing models across vendors, that neutrality is the point. The same curated suite stays reproducible no matter which provider serves the requests.

Since openbench treats a provider as an API-key environment variable plus a model string, an OrcaRouter option would let evaluators reach many chat, reasoning, and image models through a single endpoint. That is useful for benchmarking workflows that span a wide model set or run for a long time: fewer keys to manage, plus automatic provider failover when one upstream rate-limits or drops a request.

Describe the solution you'd like

I'm an engineer on the OrcaRouter team, and I'd like to propose OrcaRouter as an optional provider. It exposes an OpenAI-compatible API and uses standard API-key authentication, so it should fit openbench's existing provider layer: users would set an API-key environment variable and pass model strings like orcarouter/<model-id> to bench eval, as they already do for Groq, OpenRouter, or Together AI. The capabilities most relevant to an eval tool are:

• Many chat, reasoning, and image models reachable through one endpoint, cutting key sprawl when a benchmark spans several vendors.
• Automatic model routing and provider failover, so long, high-sample evals are not lost to a single upstream outage or rate limit.
• Prompt caching plus usage tracking and budgets, for teams that want to keep an eye on eval spend across many runs.

This is purely additive. No existing provider changes, and current benchmarks keep working. Nothing here is implemented or tested yet; I would value maintainers' guidance on the right integration point (likely the provider layer and the README provider table) before writing any code.

Describe alternatives you've considered

Teams can keep managing separate provider keys today, but this request is about consolidation: one OpenAI-compatible endpoint with routing and failover instead of N keys and manual retries. OrcaRouter is already used by open-source projects such as promptfoo, Dify, RAGFlow, and goose, which can serve as reference integrations.

Use case

An openbench user scores the same suite across many models from several vendors and wants a single credential to manage, with automatic fallback so a long run finishes without manual restarts.

Additional context

Transparency: OrcaRouter runs an optional open-source partner program in which approved OSS projects can receive a 5% revenue share from OrcaRouter usage attributed to their integration. Participation is not a prerequisite for integration, and we would defer to openbench's own disclosure and governance rules before any such discussion.

See how other projects list OrcaRouter at https://www.orcarouter.ai/built-with. I would welcome maintainers' thoughts on whether this fits openbench, and with approval I'd be glad to submit an implementation PR following the contribution guidelines.