googleapis / googleapis/python-genai

Feature request: support caching response_schema/structured-output config in CreateCachedContentConfig

Open
#2,959 1 comment 0 reactions 1 assignee Claimed by @Venkaiahbabuneelam View on GitHub
priority: p3 type: feature request
Dominant language
Python
Stars
4k
Forks
1k
Avg merge
2d 11h
Merged PRs (30d)
40

Description

`CreateCachedContentConfig` currently accepts `system_instruction`, `contents`, `tools`, and `tool_config`, but not `response_schema`/`response_mime_type`. For prompts with large structured-output schemas, this means the schema is resent at full price on every call even when `system_instruction` is cached.

The only current workaround is converting the schema into a single forced-choice `FunctionDeclaration`/`Tool` and baking that + `tool_config` into the cache instead — which we validated works (schema tokens do show up under `cached_content_token_count`), but it comes with real downsides vs. native `response_schema` support:

- Loses whatever constrained-decoding guarantee `response_schema` mode provides — function-calling isn't documented as giving the same schema-adherence guarantee.
- Introduces `MALFORMED_FUNCTION_CALL` as a new failure mode (see #1789) with no equivalent under `response_schema` mode.
- Requires a full generate_content config swap (can't mix `response_schema` with `tools`/`tool_config` in the same cached request per current API validation), so cache-hit vs. cache-miss calls need materially different request shapes.

Feature request: allow `response_schema`/`response_mime_type` to be part of the cached content directly, so structured-output prompts get the same caching benefit as system-instruction-heavy ones without the function-calling workaround.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.