googleapis / googleapis/java-genai
Unable to use explicit caching in batch mode
- Dominant language
- Java
- Stars
- 397
- Forks
- 125
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 41
Description
#### Environment details
- Programming language: Java
- OS: Windows/Linux
- Language runtime version: 21
- Package version: 1.10.0
#### Steps to reproduce
1. Create a cache for the system message
```java
CachedContent cachedContent = client.caches.create(model.getModel(),
CreateCachedContentConfig.builder()
.systemInstruction(Content.fromParts(Part.fromText(systemMessage)))
.ttl(Duration.ofDays(1L))
.build());
```
2. Create a batch job and try to use the cache
```java
String cacheName = cachedContent.name().get();
BatchJob job = client.batches.create(model.getModel(),
BatchJobSource.builder()
.inlinedRequests(InlinedRequest.builder()
.contents(contents)
.config(GenerateContentConfig.builder()
.cachedContent(cacheName)
.responseMimeType("application/json")
.responseSchema(Schema.fromJson(schema.getJsonSchema()))
.build())
.build())
.build(),
CreateBatchJobConfig.builder().build());
```
3. Run:
```
com.google.genai.errors.ClientException: 400 . Invalid JSON payload received. Unknown name "cachedContent" at 'batch.input_config.requests.requests[0]': Cannot find field.
at com.google.genai.errors.ApiException.throwFromResponse(ApiException.java:94)
at com.google.genai.HttpApiResponse.getBody(HttpApiResponse.java:37)
at com.google.genai.Batches.privateCreate(Batches.java:2384)
at com.google.genai.Batches.create(Batches.java:2684)
```
When looking into the [API](https://ai.google.dev/api/batch-mode#GenerateContentRequest) it appears that `cached_content` needs to be part of `InlinedRequest` (resp. its builder). Is this assumption correct? I looked into the Python type defintions and it does it the same way.
Contributor guide
Assessment
This issue has not been assessed yet.