ChatGPTNextWeb / ChatGPTNextWeb/NextChat

[Feature Request] Support Anthropic Model on Google Vertex

Open
#5,772 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
88.8k
Forks
59.1k
PR merge metrics
No merged PRs in 30d

Description

### 🥰 Feature Description

Currently, I can't get it to work with Claude 3.5 on Google Cloud Vertex AI. Google Cloud offers a $300 no-cost trial credit for 90 days. Could anyone help with this?

### 🧐 Proposed Solution

Here are some information from google cloud documentation:

Note that the API for Claude on Vertex differs from the Anthropic API documentation in the following ways:
- model is not a valid parameter. The model is instead specified in the Google Cloud endpoint URL.
- anthropic_version is a required parameter and must be set to "vertex-2023-10-16"

request.json
```json
{
"anthropic_version": "vertex-2023-10-16",
"messages": [
{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/png",
"data": "iVBORw0KGg..."
}
},
{
"type": "text",
"text": "What is in this image?"
}
]
}
],
"max_tokens": 256,
"stream": true
}
```
```shell
PROJECT_ID=your-gcp-project-id
MODEL=claude-3-5-sonnet-v2@20241022

# Pick one region:
LOCATION=us-east5
# LOCATION=europe-west1

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://$LOCATION-aiplatform.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION/publishers/anthropic/models/$MODEL:streamRawPredict"
```

### 📝 Additional Information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.