JetBrains / JetBrains/koog

`CachedPromptExecutor` fails to serialize the prompt cache key when a message carries a `CacheControl` directive (`Serializer for subclass 'Default' is not found in the polymorphic scope of 'CacheControl'`)

Open
#2,145 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Kotlin
Stars
4.6k
Forks
474
Avg merge
58m
Merged PRs (30d)
1

Description

## Summary

When a prompt carries a `CacheControl` directive (e.g. `AnthropicCacheControl.Default`,
set via the `PromptBuilder.system(cache = ...)` overload) and that prompt is run
through a `CachedPromptExecutor`, execution fails with a
`kotlinx.serialization.SerializationException` **before any LLM request is made**.

The failure is in the cache-key computation, which serializes the whole `Prompt`
with a `Json` that has no `SerializersModule` registered for the polymorphic
`CacheControl` interface. Because `CacheControl` is a plain (non-`@Serializable`,
non-sealed) interface and no koog module registers its subclasses, the encoder
cannot resolve a serializer for `AnthropicCacheControl.Default`.

This makes prompt caching unusable for any prompt that uses koog's own
`CacheControl` directive, regardless of the target provider/model (the error is in
cache-key serialization, not in the LLM client).

## Affected version

- **1.0.0** (latest release at time of writing).
- `develop` HEAD is identical in the relevant files (`CacheControl.kt`,
`PromptCache.kt`, `FilePromptCache.kt`), so the latest source is also affected.
- No existing issue or PR found for this.

## Environment

- koog `1.0.0`
- kotlinx-serialization `1.11.0`, Kotlin `2.3.x`, JVM target.
- Artifacts involved: `prompt-model`, `prompt-executor-cached`, `prompt-cache-files`,
`prompt-cache-model`, `prompt-executor-clients:prompt-executor-anthropic-client`.

## Minimal reproduction

Self-contained; uses only public koog API. The nested executor is a stub and is
never reached — the exception fires during cache-key computation.

```kotlin
import ai.koog.agents.core.tools.ToolDescriptor
import ai.koog.prompt.Prompt
import ai.koog.prompt.cache.files.FilePromptCache
import ai.koog.prompt.dsl.ModerationResult
import ai.koog.prompt.dsl.prompt
import ai.koog.prompt.executor.cached.CachedPromptExecutor
import ai.koog.prompt.executor.clients.anthropic.AnthropicCacheControl
import ai.koog.prompt.executor.clients.openai.OpenAIModels
import ai.koog.prompt.executor.model.PromptExecutor
import ai.koog.prompt.llm.LLModel
import ai.koog.prompt.message.Message
import ai.koog.prompt.streaming.StreamFrame
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.emptyFlow
import kotlinx.coroutines.runBlocking
import java.nio.file.Files

fun main() = runBlocking {
// A nested executor that must never be reached — the bug fires before it.
val nested = object : PromptExecutor() {
override suspend fun execute(prompt: Prompt, model: LLModel, tools: List): Message.Assistant =
error("nested executor must not be reached")
override fun executeStreaming(prompt: Prompt, model: LLModel, tools: List): Flow = emptyFlow()
override suspend fun moderate(prompt: Prompt, model: LLModel): ModerationResult = error("unused")
override fun close() = Unit
}

val executor = CachedPromptExecutor(
cache = FilePromptCache(Files.createTempDirectory("koog-cache")),
nested = nested,
)

val p = prompt("repro") {
system(cache = AnthropicCacheControl.Default) { +"You are a helpful assistant." }
user { +"What is 2 + 2?" }
}

// Throws: SerializationException — "Serializer for subclass 'Default' is not found
// in the polymorphic scope of 'CacheControl'." (model is irrelevant; never used)
executor.execute(p, OpenAIModels.Chat.GPT5_4)
}
```

### Observed exception

```
kotlinx.serialization.SerializationException: Serializer for subclass 'Default' is not found in the polymorphic scope of 'CacheControl'.
Check if class with serial name 'Default' exists and serializer is registered in a corresponding SerializersModule.
To be registered automatically, class 'Default' has to be '@Serializable', and the base class 'CacheControl' has to be sealed and '@Serializable'.
```

Top stack frames (cache-key path):

```
at kotlinx.serialization.internal.AbstractPolymorphicSerializerKt.throwSubtypeNotRegistered(...)
at ai.koog.prompt.message.MessagePart$Text.write$Self$prompt_model(Message.kt)
at ai.koog.prompt.Prompt.write$Self$prompt_model(Prompt.kt)
at ai.koog.prompt.cache.model.PromptCache$Request.(PromptCache.kt)
at ai.koog.prompt.cache.files.FilePromptCache.get(FilePromptCache.kt)
at ai.koog.prompt.executor.cached.CachedPromptExecutor.execute(CachedPromptExecutor.kt)
```

## Root cause (file:line, tag 1.0.0)

1. `ai.koog.prompt.message.CacheControl` is a **plain interface** — not
`@Serializable`, not `sealed`:
`prompt/prompt-model/src/commonMain/kotlin/ai/koog/prompt/message/CacheControl.kt:15`
```kotlin
public interface CacheControl
```

2. Message parts expose it as an **open polymorphic slot** (no `@Contextual`):
`prompt/prompt-model/.../message/Message.kt:255`
```kotlin
public val cacheControl: CacheControl?
```
(concrete overrides on `Text`/`Attachment`). With a bare-interface static type,
kotlinx.serialization treats this as polymorphic and requires a registered
subclass serializer.

3. The cache-key `Json` registers **no `serializersModule`**:
`prompt/prompt-cache/prompt-cache-model/.../cache/model/PromptCache.kt:16-19`
```kotlin
private val defaultJson = Json {
ignoreUnknownKeys = true
allowStructuredMapKeys = true
}
```
used by `Request.asCacheKey` to `encodeToString` the `@Serializable Prompt`.
(`FilePromptCache`'s own `Json` instances at `FilePromptCache.kt:22-32` are also
module-less, but the throwing encode is the one in `PromptCache.asCacheKey`.)

4. **`AnthropicCacheControl` is `@Serializable` but is never registered under the
`CacheControl` base** — and **no koog module registers `CacheControl`'s
subclasses at all** (a whole-tree search for `polymorphic(`/`subclass(` mentioning
`CacheControl` finds nothing):
`prompt-executor-clients/prompt-executor-anthropic-client/.../anthropic/AnthropicCacheControl.kt:9-18`
```kotlin
@Serializable
public sealed interface AnthropicCacheControl : CacheControl {
@Serializable public data object Default : AnthropicCacheControl
@Serializable public data object OneHour : AnthropicCacheControl
}
```

So the value is stored under the unsealed, non-`@Serializable` base `CacheControl`,
and nothing teaches any `Json` how to encode it polymorphically.

### Why only the cache path is affected

The Anthropic client never serializes the open base — it converts the directive
imperatively to an internal wire DTO before encoding
(`AnthropicLLMClient.kt`: `CacheControl.toAnthropicCacheControl()` →
`AnthropicCacheControlBlock`, `@SerialName("ephemeral")`). Only the
`CachedPromptExecutor` cache-key path serializes the raw `Prompt`, so caching is the
only place the unregistered `CacheControl` slot is hit.

## Proposed fix (any one resolves it; first is cleanest)

1. **Make `CacheControl` a `@Serializable sealed interface`** in `prompt-model`, with
the provider directives (`AnthropicCacheControl` and any others) as registered
subclasses. kotlinx.serialization then resolves the polymorphic subclass
automatically. (Requires the provider subtypes to be visible to / registered with
the base — e.g. via a `SerializersModule` aggregated where the providers are
known.)
2. **Register a `SerializersModule` on the cache-key `Json`** (`PromptCache.defaultJson`,
and the `FilePromptCache` Json instances) that declares
`polymorphic(CacheControl::class) { subclass(AnthropicCacheControl.Default); subclass(AnthropicCacheControl.OneHour); ... }`.
3. **Mark the `cacheControl` field `@Contextual`** (or give it an explicit serializer)
and provide a contextual serializer for `CacheControl` in the cache Json.

A consumer-side note (in case it's useful for triage): there is currently no public
seam to fix this from outside koog — `CachedPromptExecutor`, the `PromptCache`
interface, and `FilePromptCache` expose no `Json`/`SerializersModule` parameter, and
the failing `defaultJson` is `private` in `prompt-cache-model`. The only consumer
workaround is to avoid putting a `CacheControl` directive on cached prompts, or to
supply a fully custom `PromptCache` implementation.

## Impact

Prompt caching via `CachedPromptExecutor` is unusable for any prompt that sets a
`CacheControl` directive through koog's own `PromptBuilder.system(cache = ...)` /
message-part `cacheControl` API — a hard crash before the LLM call, on every
provider.

Contributor guide

Open the contributing guide

Research direction

Run the minimal reproduction, then inspect CacheControl.kt, Message.kt, PromptCache.kt, FilePromptCache.kt, and AnthropicCacheControl.kt to trace the cache-key serialization path. Choose and implement a serialization approach that covers provider directives, then verify that CachedPromptExecutor reaches the nested executor without a SerializationException when a cache directive is present.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.