googleapis / googleapis/python-genai

Vertex AI: unable to use explicit context cache with fine-tuned model

Open
#1,693 12 comments 0 reactions 1 assignee Claimed by @Venkaiahbabuneelam View on GitHub
priority: p2 type: bug
Dominant language
Python
Stars
4k
Forks
1k
Avg merge
2d 11h
Merged PRs (30d)
40

Description

Google's docs say you can use an explicit context cache with fine-tuned models: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-for-tuned-gemini

However, following the instructions in this page to create a context cache with a fine-tuned model (in this case, tuned from gemini-2.5-flash-lite, but same issue occurs with any base model) always gives this error for the `caches.create` call:

```
{
"error": {
"code": 501,
"message": "Explicit cache is not supported for fined tuned `projects/840939471922/locations/us-central1/models/8676953240757075968@1`.",
"status": "UNIMPLEMENTED"
}
}
```

Any ideas what could be wrong, and/or have *working* code demonstrating how to create a context cache for a fine-tuned model?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.