0xPlaygrounds / 0xPlaygrounds/rig

Architecture: make provider integrations profile- and protocol-driven

Abierto
#2,042 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Rust
Estrellas
8.6k
Forks
959
Merge medio
4 h 32 min
PR fusionados (30 d)
117

Descripción

## Context

PR #2040 exposed how difficult it is to maintain provider integrations when each provider encodes protocol and dialect differences through ad hoc request/response code. During review, the most problematic providers were not necessarily unsupported providers, but providers whose APIs are "OpenAI-compatible" only with caveats: Perplexity, Mira, Hyperbolic, Llamafile, OpenRouter, Mistral, Moonshot, and Azure.

Separately, Galadriel appears to be a dead/low-confidence provider and is being removed in #2041.

## Research notes

I reviewed the current Rig provider architecture and compared it against several inspiration repositories under `many_rigs/inspirations`:

- Vercel AI SDK
- Pydantic AI
- LangChain
- OpenAI Agents Python

Useful patterns found:

### Vercel AI SDK

- Separates provider interfaces from concrete provider packages.
- Uses common model interfaces and provider-specific packages.
- Has middleware such as `wrapLanguageModel` and default-settings transforms.
- Has provider-specific options rather than one global arbitrary params bag.

Relevant ideas for Rig:

- Language/model middleware for request transforms.
- Provider-specific options namespace.
- Thin provider wrappers around protocol implementations.

### Pydantic AI

- Separates `Provider` from `Model`.
- Uses `ModelProfile` to describe model/provider capabilities:
- tools
- structured output
- image support
- thinking/reasoning
- system prompt behavior
- native tool support
- Has wrapper/fallback models.

Relevant ideas for Rig:

- Introduce `ModelProfile` / `ProviderProfile` instead of scattering constants and hooks across provider impls.
- Profiles should describe behavior independent of concrete client code.

### LangChain

- Keeps many provider integrations in partner packages rather than core.
- Has standard integration test suites.
- Has model profile data generated/updated from external sources such as `models.dev`.

Relevant ideas for Rig:

- Move concrete providers out of `rig-core` over time.
- Add standard provider conformance tests.
- Maintain model capability/profile data in a structured way.

## Proposed architecture direction

Rig should split provider handling into explicit layers:

```text
Rig request / agent layer

Canonical model request

Capability/profile negotiation

Protocol adapter
- OpenAI Chat Completions
- OpenAI Responses
- Anthropic Messages
- Cohere native
- Gemini native

Provider dialect/profile
- Azure route format
- Mistral tool_choice = any
- Perplexity no tools
- Hyperbolic no tools/schema
- OpenRouter reasoning/files/images

HTTP client/auth/base URL
```

## Specific improvements

### 1. Add `ModelProfile` / `ProviderProfile`

Instead of encoding provider differences as scattered trait constants and JSON hooks, introduce a profile object, e.g.:

```rust
pub struct ModelProfile {
pub supports_tools: bool,
pub supports_tool_choice_required: bool,
pub supports_tool_choice_specific: bool,
pub supports_json_schema_output: bool,
pub supports_json_object_output: bool,
pub supports_stream_usage: StreamUsageMode,
pub supports_reasoning: bool,
pub supports_images: bool,
pub supports_documents: bool,
pub requires_string_content: bool,
pub requires_role_alternation: bool,
pub default_structured_output_mode: StructuredOutputMode,
}
```

Example provider profile:

```rust
impl ProviderProfile for Perplexity {
fn profile(&self, model: &str) -> ModelProfile {
ModelProfile {
supports_tools: false,
supports_json_schema_output: false,
requires_string_content: true,
requires_role_alternation: true,
..Default::default()
}
}
}
```

This would make issues like the PR #2040 Perplexity/Mira/Hyperbolic regressions much more visible.

### 2. Separate protocol adapters from provider wrappers

Create explicit protocol modules:

```text
protocols/
openai_chat/
openai_responses/
anthropic_messages/
cohere/
gemini/
```

Then concrete providers become thin configurations over protocols:

```text
providers/openai.rs -> OpenAI Chat/Responses protocol
providers/azure.rs -> OpenAI protocol + Azure route/profile
providers/hyperbolic.rs -> OpenAI protocol + no-tools/no-schema profile
providers/moonshot.rs -> OpenAI + Anthropic protocols + Kimi quirks
providers/zai.rs -> OpenAI + Anthropic protocols
```

### 3. Replace ad hoc hooks with named request/response transforms

Today `OpenAICompatibleProvider` mixes multiple concerns:

- telemetry provider name
- route construction
- tool support
- structured-output support
- streaming usage type
- typed request mutation
- serialized JSON mutation
- response type

Consider replacing generic `prepare_request` / `finalize_request_body` hooks with named transforms:

```rust
pub enum RequestTransform {
DropUnsupportedTools,
FlattenContentParts,
EnforceRoleAlternation,
RewriteToolChoiceRequiredToAny,
StripReasoningContent,
MergeStreamOptions,
ApplyResponseFormat,
}
```

Providers would declare their transforms:

```rust
const TRANSFORMS: &[RequestTransform] = &[
DropUnsupportedTools,
FlattenContentParts,
EnforceRoleAlternation,
];
```

This would be easier to inspect, test, and compose than provider-local JSON mutation.

### 4. Namespace provider-specific options

`additional_params: serde_json::Value` is too easy to misuse. It caused conflicts such as provider-native Groq tools colliding with standard `tools`.

Consider a provider options structure or namespace:

```rust
.additional_provider_params("groq", json!({ ... }))
```

or typed provider options:

```rust
ProviderOptions {
openai: OpenAiOptions,
groq: GroqOptions,
openrouter: OpenRouterOptions,
mistral: MistralOptions,
}
```

### 5. Add standard provider conformance tests

Borrow LangChain's `standard-tests` idea.

Create provider contract tests for:

```text
completion_basic
streaming_basic
tools
tool_choice
structured_output
documents
images
reasoning
error_preservation
usage
```

Providers should opt into a capability matrix:

```rust
provider_contract_tests!(Hyperbolic {
completion: true,
streaming: true,
tools: false,
structured_output: false,
});
```

Then CI can verify that the profile and actual behavior agree.

### 6. Eventually move concrete providers out of `rig-core`

Long-term structure:

```text
crates/
rig-core/
protocols/
traits/
agent/
tool/
vector_store/

rig-provider-openai/
rig-provider-anthropic/
rig-provider-cohere/
rig-provider-perplexity/
rig-provider-hyperbolic/
```

The root `rig` crate can preserve ergonomics through feature-gated re-exports:

```rust
rig::providers::openai
rig::providers::perplexity
```

## Recommended rollout

1. Remove dead/low-confidence providers first.
- Galadriel is already proposed for removal in #2041.
- Mira should be live-audited next.

2. Add `ModelProfile` / `ProviderProfile`.

3. Convert OpenAI-compatible providers to profile-driven behavior.

4. Replace ad hoc hooks with named transforms.

5. Add provider conformance tests.

6. Move provider crates out of `rig-core` once the interfaces are stable.

## Design principle

Rig should stop asking only:

> Is this provider OpenAI-compatible?

and instead ask:

> Which protocol does this provider speak, and which capabilities/dialect transforms does this specific model require?

This should make provider additions and migrations much safer than PR #2040 was.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.