googleapis / googleapis/python-genai
Feature request: support caching response_schema/structured-output config in CreateCachedContentConfig
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
`CreateCachedContentConfig` currently accepts `system_instruction`, `contents`, `tools`, and `tool_config`, but not `response_schema`/`response_mime_type`. For prompts with large structured-output schemas, this means the schema is resent at full price on every call even when `system_instruction` is cached.
The only current workaround is converting the schema into a single forced-choice `FunctionDeclaration`/`Tool` and baking that + `tool_config` into the cache instead — which we validated works (schema tokens do show up under `cached_content_token_count`), but it comes with real downsides vs. native `response_schema` support:
- Loses whatever constrained-decoding guarantee `response_schema` mode provides — function-calling isn't documented as giving the same schema-adherence guarantee.
- Introduces `MALFORMED_FUNCTION_CALL` as a new failure mode (see #1789) with no equivalent under `response_schema` mode.
- Requires a full generate_content config swap (can't mix `response_schema` with `tools`/`tool_config` in the same cached request per current API validation), so cache-hit vs. cache-miss calls need materially different request shapes.
Feature request: allow `response_schema`/`response_mime_type` to be part of the cached content directly, so structured-output prompts get the same caching benefit as system-instruction-heavy ones without the function-calling workaround.
Contributor guide
Assessment
This issue has not been assessed yet.