googleapis / googleapis/java-genai

Support to access model gardens using endpoints: rawPredict or streamRawPredict

Open
#900 4 comments 1 reaction 1 assignee Claimed by @hemasekhar-p View on GitHub
priority: p3 type: feature request
Dominant language
Java
Stars
397
Forks
125
Avg merge
2d 8h
Merged PRs (30d)
41

Description

## Description

The Vertex AI documentation describes that we can select one of the different models: open, partner, gemini-family and access them using [curl](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/partner-models/claude/use-claude#use_a_curl_command), python and probably this Java SDK (as documented [here](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/start/libraries)) but when we use the following code

```java
Client client = null;
try {
GoogleCredentials creds = GoogleCredentials.getApplicationDefault();
creds.refreshIfExpired();

client = Client.builder()
.credentials(creds)
.build();
} catch (Exception e) {
throw new RuntimeException(e);
}

GenerateContentResponse response =
client.models.generateContent("anthropic/claude-opus-4-6", "What is your name?", null);
```
then we got this error
```
com.google.genai.errors.ClientException: 400 . claude-opus-4-6 is not supported in the generateContent API.
at com.google.genai.errors.ApiException.throwFromResponse (ApiException.java:94)
at com.google.genai.HttpApiResponse.getBody (HttpApiResponse.java:37)
at com.google.genai.Models.processResponseForPrivateGenerateContent (Models.java:5480)
at com.google.genai.Models.privateGenerateContent (Models.java:5546)
at com.google.genai.Models.generateContent (Models.java:7076)
at com.google.genai.Models.generateContent (Models.java:7150)
at com.google.genai.examples.GenerateContent.main (GenerateContent.java:89)
```

## Suggestion

This Java SDK project is a great library as it allows third-party projects to access the Google Cloud platform and Vertex AI using HTTP(s) with JSON body messages instead of using gRPC.

As the access to one of the model partners like Claude, Mistral, etc relies on the following endpoints
```
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:rawPredict
https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:streamRawPredict
```
I suggest that this Java SDK also support them. Here is the API documentation: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/reference/rest/v1/projects.locations.endpoints/rawPredict

FYI: The projects [Langchain4j](https://github.com/langchain4j/langchain4j/tree/main/langchain4j-vertex-ai) and [Quarkus Langchains4j](https://github.com/quarkiverse/quarkus-langchain4j/tree/main/model-providers/google/vertex-ai) currently use the gRPC code of the following GAV and will be more than happy to use now this SDK with HTTP(S).
```

com.google.cloud
google-cloud-aiplatform
3.69.0

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.