Kaggle / Kaggle/kaggle-benchmarks
[feature request]: Support prompt caching parameters
- Dominant language
- Python
- Stars
- 190
- Forks
- 53
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 3
Description
### Bug Description
This is not exactly a bug, but a feature request/infrastructure support request.
I tested prompt caching behavior across different model providers using kaggle-benchmarks. My main finding is that GPT and Qwen appear to benefit from prompt caching, which suggests prompt caching could significantly reduce costs for long-running agent benchmark tasks.
However, other providers I tested, including Anthropic and Google/Gemini, did not appear to benefit from caching in the same way. This may be because provider-specific prompt caching parameters are not currently supported or forwarded through the kaggle-benchmarks model proxy and/or SDK.
This matters because prompt caching can substantially affect both reported cost and latency. Without clear support or visibility into caching behavior, users may incorrectly conclude that one model is faster or cheaper than another, when the difference may actually come from whether prompt caching is being applied.
### Kaggle Username
anhoangvo
### Execution Environment
Kaggle Notebook
### Are you using Kaggle Benchmarks in your code?
Yes — I import and use kaggle_benchmarks directly
### Steps to Reproduce
1. Open the shared Kaggle notebook: https://www.kaggle.com/code/anhoangvo/prompt-cache-tester-kaggle-benchmarks
2. Run notebook.
3. Analyze the results.
### Code to Reproduce
```python
```
### Expected Behavior
I expected prompt caching support to be consistently available where the underlying provider supports it, either automatically or through provider-specific parameters.
In particular, for providers that require explicit cache parameters, I expected kaggle-benchmarks or its model proxy to preserve and forward those parameters when possible.
Examples of provider-specific caching behavior I found:
- OpenAI / xAI / DeepSeek / Gemini: prompt caching is generally automatic, with cached token information exposed through usage metadata such as `usage.prompt_tokens_details.cached_tokens`.
- DeepSeek: automatic, but uses fields such as `usage.prompt_cache_hit_tokens` and `usage.prompt_cache_miss_tokens`.
- Anthropic: requires explicit `cache_control: {"type": "ephemeral"}` on the content block; otherwise, caching does not occur.
- Qwen: appears to support `cache_control: {"type": "ephemeral"}` using Anthropic-like syntax, but only for selected models.
- ZAI / GLM-5: I could not find public caching documentation, so I treated it as automatic.
- xAI: automatic, but recommends using the `x-grok-conv-id` header per conversation to maximize cache hits.
### Actual Behavior
Prompt caching appears to work for some providers/models, such as GPT and Qwen, but not consistently across other providers.
For providers that require explicit caching parameters, such as Anthropic, caching may effectively be disabled if the kaggle-benchmarks model proxy strips, modifies, or fails to forward provider-specific caching fields, such as `cache_control`.
This makes cost and latency comparisons potentially misleading across providers.
### Error Output / Traceback
```shell
```
### Environment Info
```shell
```
### LLM Model Used
_No response_
### Shared Asset Links
https://www.kaggle.com/code/anhoangvo/prompt-cache-tester-kaggle-benchmarks
### Additional Context
_No response_
### Pre-submission Checklist
- [x] I have searched the existing issues and this bug has not been reported before.
- [x] I am using the latest version of kaggle-benchmarks (or have noted my version above).
- [x] I have shared my relevant Kaggle assets (notebooks, datasets, etc.) with kaggle-ai-resources-support@google.com as a Viewer, or I have explained in the issue why I cannot.
Contributor guide
Research direction
Start by tracing how provider request fields and usage metadata move through the kaggle-benchmarks model proxy and SDK, then run the linked Kaggle notebook to compare providers. Done should include a documented, tested approach for preserving supported provider-specific caching parameters and exposing enough cache usage information for cost and latency comparisons.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100