0xPlaygrounds / 0xPlaygrounds/rig

feat: Support Anthropic/Claude via Google Vertex AI

Open
#1,598 1 comment 2 reactions 0 assignees View on GitHub
feat
Dominant language
Rust
Stars
8.6k
Forks
959
Avg merge
4h 32m
Merged PRs (30d)
117

Description

- [x] I have looked for existing issues (including closed) about this

## Feature Request
Google Vertex AI also serves third-party models such as Anthropic's Claude models.
Unfortunately, it looks like the library currently does not support this via the `rig-vertexai` crate.

## Current behavior
When passing one of the Claude model strings (instead of Gemini) to the `rig-vertexai` client's `completion_model()` function, a 404 error (model not found) is returned by Google Vertex AI when sending the request. Internally, [the URL is hardcoded](https://github.com/0xPlaygrounds/rig/blob/12227949bec82b1d25772fb7044aff1e58e463c5/rig-integrations/rig-vertexai/src/completion.rs#L61-L69) to the `google` publisher.

However, even when hardcoding the publisher to `anthropic`, a 404 error is produced.
This error by Google Vertex AI is actually misleading. The issue is that `rig-vertexai` talks to the [`generateContent` endpoint](https://github.com/0xPlaygrounds/rig/blob/12227949bec82b1d25772fb7044aff1e58e463c5/rig-integrations/rig-vertexai/src/completion.rs#L103) which is not implemented for Anthropic models.
Instead, the `rawPredict`/`streamRawPredict` endpoint should be used which are [very close to the plain Anthropic API](https://platform.claude.com/docs/en/build-with-claude/claude-on-vertex-ai).

### Proposal
Implement a dedicated client struct (e.g., `AnthropicClient`) in `rig-vertexai` which accepts Anthropic model names as hosted on Vertex AI. Contrary to the main `Client` in `rig-vertexai`, it uses the `rawPredict` endpoint.
[Code/Types/JSON Serialization from the `anthropic` provider](https://github.com/0xPlaygrounds/rig/tree/main/rig/rig-core/src/providers/anthropic) can most probably be reused.

### Alternatives
- Implement Anthropic Vertex AI support in the main `rig-vertexai` `Client` struct and dynamically determine based on the model name how to call the API. This provides for a cleaner interface but might be too much magic.
- Similar but implement the support by modifying the `rig-vertexai` `CompletionModel` struct. There could be an explicit `::anthropic` constructor which must be used instead of the standard constructor.
- Introduce a layer of abstraction that makes the user of `rig-vertexai` explicitly specify which publisher in Vertex AI to use, including explicitly specifying Google as publisher. This is probably the cleanest solution, considering that Vertex AI also hosts other third-party models, but would break the API of `rig-vertexai`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.