ChatGPTNextWeb / ChatGPTNextWeb/NextChat
[Feature Request] Support Anthropic Model on Google Vertex
- Dominant language
- TypeScript
- Stars
- 88.8k
- Forks
- 59.1k
- PR merge metrics
- No merged PRs in 30d
Description
### 🥰 Feature Description
Currently, I can't get it to work with Claude 3.5 on Google Cloud Vertex AI. Google Cloud offers a $300 no-cost trial credit for 90 days. Could anyone help with this?
### 🧐 Proposed Solution
Here are some information from google cloud documentation:
Note that the API for Claude on Vertex differs from the Anthropic API documentation in the following ways:
- model is not a valid parameter. The model is instead specified in the Google Cloud endpoint URL.
- anthropic_version is a required parameter and must be set to "vertex-2023-10-16"
request.json
```json
{
"anthropic_version": "vertex-2023-10-16",
"messages": [
{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/png",
"data": "iVBORw0KGg..."
}
},
{
"type": "text",
"text": "What is in this image?"
}
]
}
],
"max_tokens": 256,
"stream": true
}
```
```shell
PROJECT_ID=your-gcp-project-id
MODEL=claude-3-5-sonnet-v2@20241022
# Pick one region:
LOCATION=us-east5
# LOCATION=europe-west1
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://$LOCATION-aiplatform.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION/publishers/anthropic/models/$MODEL:streamRawPredict"
```
### 📝 Additional Information
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.