anomalyco / anomalyco/opencode

[Bug]: Gemini models on OpenCode Zen return zero cached tokens for repeated prompt prefixes

Open
#45,049 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
209k
Forks
27.5k
PR merge metrics
PR metrics pending

Description

Description

Description

This report concerns the OpenCode Zen HTTP API itself, not the OpenCode client.

Gemini models on OpenCode Zen appear to never hit the server-side implicit prompt cache.

I reproduced this by sending two consecutive requests directly to the Zen Gemini generateContent endpoint. Both requests contain a large identical prefix and differ only in a short suffix.

The second request still reports:

cachedContentTokenCount = 0

and is billed entirely at the normal input-token rate.

Reproduction

Tested with:

gemini-3.7-flash
gemini-3.1-pro

Minimal Python repro:

import time
import requests

API_KEY = "REDACTED"
MODEL = "gemini-3.7-flash"

url = (
    f"https://opencode.ai/zen/v1/models/"
    f"{MODEL}:generateContent"
)

headers = {
    "x-goog-api-key": API_KEY,
    "Content-Type": "application/json",
}

# Large stable prefix, well above the implicit cache minimum.
stable_prefix = (
    """
This is fixed prompt-cache test content.
The contents of this block are identical between requests.
ABCDEFGHIJKLMNOPQRSTUVWXYZ
0123456789
The quick brown fox jumps over the lazy dog.
"""
    * 1200
)


def run_test(suffix):
    payload = {
        "contents": [
            {
                "role": "user",
                "parts": [
                    {
                        "text": (
                            stable_prefix
                            + "\n"
                            + suffix
                            + "\nReply only with: OK"
                        )
                    }
                ],
            }
        ],
        "generationConfig": {
            "temperature": 0,
            "maxOutputTokens": 128,
        },
    }

    r = requests.post(
        url,
        headers=headers,
        json=payload,
        timeout=120,
    )

    r.raise_for_status()
    data = r.json()

    usage = data.get("usageMetadata", {})

    print(
        "prompt=",
        usage.get("promptTokenCount"),
        "cached=",
        usage.get("cachedContentTokenCount", 0),
        "cost=",
        data.get("cost"),
    )


run_test("ROUND_ONE")
time.sleep(2)
run_test("ROUND_TWO")

Expected behavior:

First request:
cached = 0

Second request:
cached > 0

Actual behavior:

First request:
cached = 0

Second request:
cached = 0

I also observed the same behavior during normal usage with gemini-3.1-pro.

Example consecutive requests:

model=gemini-3.1-pro
promptTokenCount=21750
cachedContentTokenCount=0
outputTokenCount=1922
cost=$0.066564

model=gemini-3.1-pro
promptTokenCount=22787
cachedContentTokenCount=0
outputTokenCount=1407
cost=$0.062458

The billing matches full uncached input pricing.

Why this seems unexpected

Gemini supports server-side implicit caching for sufficiently large repeated prompt prefixes.

Zen's pricing page also lists a separate Cached Read price for Gemini models.

There is also existing discussion in the OpenCode repository showing native Gemini/Vertex requests successfully receiving very high cached-token counts on repeated prefixes.

Question

Is implicit prompt caching currently supported for Gemini models through OpenCode Zen?

If cachedContentTokenCount = 0 is expected for Zen Gemini requests, it would be useful to document that explicitly, because the pricing page currently lists Cached Read pricing for these models.

Plugins

N/A

OpenCode version

N/A — reproduced directly against the OpenCode Zen HTTP API

Steps to reproduce
  1. Create an OpenCode Zen API key with available credit.
  2. Send a generateContent request to gemini-3.7-flash containing a prompt significantly larger than the implicit-cache minimum.
  3. Immediately send a second request with the same large prefix, changing only a short suffix at the end.
  4. Inspect usageMetadata.cachedContentTokenCount in both responses.
  5. Observe that the second response still reports cachedContentTokenCount: 0.
  6. The behavior is also reproducible with gemini-3.1-pro.
Screenshot and/or share link

No response

Operating System

N/A

Terminal

N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided Python reproduction against the OpenCode Zen Gemini generateContent endpoint, using the consecutive requests and usageMetadata.cachedContentTokenCount output. Check whether Zen's Gemini integration supports implicit caching and compare the observed behavior with the pricing page's Cached Read listing. Done means the support status and expected billing behavior are confirmed and clearly documented or the service issue is identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.