google-research / google-research/android_world
OrcaRouter provider support for AndroidWorld
- Dominant language
- Python
- Stars
- 906
- Forks
- 186
- Avg merge
- 1h 21m
- Merged PRs (30d)
- 1
Description
**Proposal: Add OrcaRouter as an optional OpenAI-compatible model provider for AndroidWorld**
AndroidWorld gives the computer-control-agent community a benchmark it can trust: 116 hand-crafted tasks across 20 apps, dynamically instantiated with randomly generated parameters so every run exercises fresh variations, plus durable reward signals for comparable results. For researchers deciding whether an agent generalizes, that reproducibility turns a leaderboard entry into evidence.
The project already makes model backends a swappable choice: you export provider keys as environment variables, and each provider is a small wrapper - Gpt4Wrapper reads OPENAI_API_KEY and calls the OpenAI chat-completions endpoint directly; GeminiGcpWrapper reads GCP_API_KEY. Your README's "Note on Model Cost" even suggests switching `model_name` to a cheaper model, which is exactly the workflow where more provider options are low-cost to add and immediately useful for large evaluation sweeps.
I'm an engineer on the OrcaRouter team, and I'd like to propose adding OrcaRouter as an optional provider alongside the existing ones - it would not replace or change any current provider. OrcaRouter exposes an OpenAI-compatible API and uses standard API-key authentication, so the natural integration point mirrors what Gpt4Wrapper already does: read a new environment variable and point an LlmWrapper-style client at OrcaRouter's base URL as another selectable model backend. Nothing is implemented or tested yet; this issue is to gauge whether maintainers are open to the idea and how they'd like it scoped.
The capabilities most relevant to AndroidWorld users would be: a single endpoint covering many chat, reasoning, image, and video models, so a benchmark sweep across model families needs no new wrapper per provider; automatic routing with provider failover, which helps long suite runs survive one provider's outage; prompt caching, since the same task instructions recur across hundreds of parameter variations; and usage tracking with budgets, which speaks directly to the model-cost concern in your README.
OrcaRouter is still new in the open-source ecosystem, but it is already landing as an optional provider in projects such as Dify, promptfoo, and CAMEL.
Transparency note, since governance matters for a Google project: OrcaRouter runs an optional open-source partner program in which approved OSS projects can receive a 5% revenue share from OrcaRouter usage attributed to their integration. Participation is not a condition for integrating - the provider behaves the same without it - and I'd be happy to follow this project's disclosure and governance requirements. You can see how other projects present it at https://www.orcarouter.ai/built-with.
Would you be open to this direction? If so, I'd be glad to prepare an implementation PR following the existing wrapper pattern, per the CONTRIBUTING guidelines.
Contributor guide
Research direction
Review the existing Gpt4Wrapper and GeminiGcpWrapper patterns, the LlmWrapper-style client entry point, and the README's "Note on Model Cost" alongside CONTRIBUTING. The issue is currently a proposal seeking maintainer direction; completion would require an agreed scope for an optional provider and corresponding tests, which are not specified here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100