googleapis / googleapis/java-genai

Unable to use explicit caching in batch mode

Open
#446 5 comments 0 reactions 1 assignee Claimed by @hemasekhar-p View on GitHub
priority: p2 type: bug
Dominant language
Java
Stars
397
Forks
125
Avg merge
2d 8h
Merged PRs (30d)
41

Description

#### Environment details

- Programming language: Java
- OS: Windows/Linux
- Language runtime version: 21
- Package version: 1.10.0

#### Steps to reproduce

1. Create a cache for the system message
```java
CachedContent cachedContent = client.caches.create(model.getModel(),
CreateCachedContentConfig.builder()
.systemInstruction(Content.fromParts(Part.fromText(systemMessage)))
.ttl(Duration.ofDays(1L))
.build());
```
2. Create a batch job and try to use the cache
```java
String cacheName = cachedContent.name().get();
BatchJob job = client.batches.create(model.getModel(),
BatchJobSource.builder()
.inlinedRequests(InlinedRequest.builder()
.contents(contents)
.config(GenerateContentConfig.builder()
.cachedContent(cacheName)
.responseMimeType("application/json")
.responseSchema(Schema.fromJson(schema.getJsonSchema()))
.build())
.build())
.build(),
CreateBatchJobConfig.builder().build());
```
3. Run:
```
com.google.genai.errors.ClientException: 400 . Invalid JSON payload received. Unknown name "cachedContent" at 'batch.input_config.requests.requests[0]': Cannot find field.
at com.google.genai.errors.ApiException.throwFromResponse(ApiException.java:94)
at com.google.genai.HttpApiResponse.getBody(HttpApiResponse.java:37)
at com.google.genai.Batches.privateCreate(Batches.java:2384)
at com.google.genai.Batches.create(Batches.java:2684)
```

When looking into the [API](https://ai.google.dev/api/batch-mode#GenerateContentRequest) it appears that `cached_content` needs to be part of `InlinedRequest` (resp. its builder). Is this assumption correct? I looked into the Python type defintions and it does it the same way.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.