0xPlaygrounds / 0xPlaygrounds/rig
Architecture: make provider integrations profile- and protocol-driven
- Lenguaje dominante
- Rust
- Estrellas
- 8.6k
- Forks
- 959
- Merge medio
- 4 h 32 min
- PR fusionados (30 d)
- 117
Descripción
## Context
PR #2040 exposed how difficult it is to maintain provider integrations when each provider encodes protocol and dialect differences through ad hoc request/response code. During review, the most problematic providers were not necessarily unsupported providers, but providers whose APIs are "OpenAI-compatible" only with caveats: Perplexity, Mira, Hyperbolic, Llamafile, OpenRouter, Mistral, Moonshot, and Azure.
Separately, Galadriel appears to be a dead/low-confidence provider and is being removed in #2041.
## Research notes
I reviewed the current Rig provider architecture and compared it against several inspiration repositories under `many_rigs/inspirations`:
- Vercel AI SDK
- Pydantic AI
- LangChain
- OpenAI Agents Python
Useful patterns found:
### Vercel AI SDK
- Separates provider interfaces from concrete provider packages.
- Uses common model interfaces and provider-specific packages.
- Has middleware such as `wrapLanguageModel` and default-settings transforms.
- Has provider-specific options rather than one global arbitrary params bag.
Relevant ideas for Rig:
- Language/model middleware for request transforms.
- Provider-specific options namespace.
- Thin provider wrappers around protocol implementations.
### Pydantic AI
- Separates `Provider` from `Model`.
- Uses `ModelProfile` to describe model/provider capabilities:
- tools
- structured output
- image support
- thinking/reasoning
- system prompt behavior
- native tool support
- Has wrapper/fallback models.
Relevant ideas for Rig:
- Introduce `ModelProfile` / `ProviderProfile` instead of scattering constants and hooks across provider impls.
- Profiles should describe behavior independent of concrete client code.
### LangChain
- Keeps many provider integrations in partner packages rather than core.
- Has standard integration test suites.
- Has model profile data generated/updated from external sources such as `models.dev`.
Relevant ideas for Rig:
- Move concrete providers out of `rig-core` over time.
- Add standard provider conformance tests.
- Maintain model capability/profile data in a structured way.
## Proposed architecture direction
Rig should split provider handling into explicit layers:
```text
Rig request / agent layer
↓
Canonical model request
↓
Capability/profile negotiation
↓
Protocol adapter
- OpenAI Chat Completions
- OpenAI Responses
- Anthropic Messages
- Cohere native
- Gemini native
↓
Provider dialect/profile
- Azure route format
- Mistral tool_choice = any
- Perplexity no tools
- Hyperbolic no tools/schema
- OpenRouter reasoning/files/images
↓
HTTP client/auth/base URL
```
## Specific improvements
### 1. Add `ModelProfile` / `ProviderProfile`
Instead of encoding provider differences as scattered trait constants and JSON hooks, introduce a profile object, e.g.:
```rust
pub struct ModelProfile {
pub supports_tools: bool,
pub supports_tool_choice_required: bool,
pub supports_tool_choice_specific: bool,
pub supports_json_schema_output: bool,
pub supports_json_object_output: bool,
pub supports_stream_usage: StreamUsageMode,
pub supports_reasoning: bool,
pub supports_images: bool,
pub supports_documents: bool,
pub requires_string_content: bool,
pub requires_role_alternation: bool,
pub default_structured_output_mode: StructuredOutputMode,
}
```
Example provider profile:
```rust
impl ProviderProfile for Perplexity {
fn profile(&self, model: &str) -> ModelProfile {
ModelProfile {
supports_tools: false,
supports_json_schema_output: false,
requires_string_content: true,
requires_role_alternation: true,
..Default::default()
}
}
}
```
This would make issues like the PR #2040 Perplexity/Mira/Hyperbolic regressions much more visible.
### 2. Separate protocol adapters from provider wrappers
Create explicit protocol modules:
```text
protocols/
openai_chat/
openai_responses/
anthropic_messages/
cohere/
gemini/
```
Then concrete providers become thin configurations over protocols:
```text
providers/openai.rs -> OpenAI Chat/Responses protocol
providers/azure.rs -> OpenAI protocol + Azure route/profile
providers/hyperbolic.rs -> OpenAI protocol + no-tools/no-schema profile
providers/moonshot.rs -> OpenAI + Anthropic protocols + Kimi quirks
providers/zai.rs -> OpenAI + Anthropic protocols
```
### 3. Replace ad hoc hooks with named request/response transforms
Today `OpenAICompatibleProvider` mixes multiple concerns:
- telemetry provider name
- route construction
- tool support
- structured-output support
- streaming usage type
- typed request mutation
- serialized JSON mutation
- response type
Consider replacing generic `prepare_request` / `finalize_request_body` hooks with named transforms:
```rust
pub enum RequestTransform {
DropUnsupportedTools,
FlattenContentParts,
EnforceRoleAlternation,
RewriteToolChoiceRequiredToAny,
StripReasoningContent,
MergeStreamOptions,
ApplyResponseFormat,
}
```
Providers would declare their transforms:
```rust
const TRANSFORMS: &[RequestTransform] = &[
DropUnsupportedTools,
FlattenContentParts,
EnforceRoleAlternation,
];
```
This would be easier to inspect, test, and compose than provider-local JSON mutation.
### 4. Namespace provider-specific options
`additional_params: serde_json::Value` is too easy to misuse. It caused conflicts such as provider-native Groq tools colliding with standard `tools`.
Consider a provider options structure or namespace:
```rust
.additional_provider_params("groq", json!({ ... }))
```
or typed provider options:
```rust
ProviderOptions {
openai: OpenAiOptions,
groq: GroqOptions,
openrouter: OpenRouterOptions,
mistral: MistralOptions,
}
```
### 5. Add standard provider conformance tests
Borrow LangChain's `standard-tests` idea.
Create provider contract tests for:
```text
completion_basic
streaming_basic
tools
tool_choice
structured_output
documents
images
reasoning
error_preservation
usage
```
Providers should opt into a capability matrix:
```rust
provider_contract_tests!(Hyperbolic {
completion: true,
streaming: true,
tools: false,
structured_output: false,
});
```
Then CI can verify that the profile and actual behavior agree.
### 6. Eventually move concrete providers out of `rig-core`
Long-term structure:
```text
crates/
rig-core/
protocols/
traits/
agent/
tool/
vector_store/
rig-provider-openai/
rig-provider-anthropic/
rig-provider-cohere/
rig-provider-perplexity/
rig-provider-hyperbolic/
```
The root `rig` crate can preserve ergonomics through feature-gated re-exports:
```rust
rig::providers::openai
rig::providers::perplexity
```
## Recommended rollout
1. Remove dead/low-confidence providers first.
- Galadriel is already proposed for removal in #2041.
- Mira should be live-audited next.
2. Add `ModelProfile` / `ProviderProfile`.
3. Convert OpenAI-compatible providers to profile-driven behavior.
4. Replace ad hoc hooks with named transforms.
5. Add provider conformance tests.
6. Move provider crates out of `rig-core` once the interfaces are stable.
## Design principle
Rig should stop asking only:
> Is this provider OpenAI-compatible?
and instead ask:
> Which protocol does this provider speak, and which capabilities/dialect transforms does this specific model require?
This should make provider additions and migrations much safer than PR #2040 was.
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.