code-yeongyu / code-yeongyu/senpi

bug(prompt-cache): direct DeepSeek is treated as a fixed 5m TTL, forcing unnecessary 4m30 Goal cache-warm wakes

Open
#831 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
429
Forks
98
Avg merge
5h 3m
Merged PRs (30d)
526

Description

## Summary

Direct DeepSeek API sessions are currently treated as if they had a deterministic 5-minute prompt-cache TTL. That propagates into Goal monitor scheduling and causes a continuation every 270 seconds (5m minus the default 30s safety buffer), with UI copy claiming the wake stays inside a 5m TTL.

DeepSeek's official context cache does **not** expose a fixed 5-minute TTL. Its disk cache is automatic, best-effort, and unused cache entries are generally cleared only after **a few hours to a few days**.

This looks like a cache-semantics classification bug rather than a DeepSeek-specific tuning request.

## Observed behavior

On a direct DeepSeek session, Senpi renders a cache-warm wait like:

```text
⚡ Cache-warm wait · iteration 1 · 1 wake source on duty
Continuation deferred 4m 30s - the timed wake stays inside the 5m prompt-cache TTL.
~9.5M tokens kept warm · est. $1.31 saved vs a cold re-read
```

The 4m30 timing is consistent with the behavior introduced in #767:

- 5-minute cache lane -> 270s Goal monitor continuation
- 1-hour cache lane -> 3570s
- unknown budget -> 240s fallback

## Why this appears incorrect for DeepSeek

DeepSeek's official docs describe context caching as:

- enabled automatically for all users
- best-effort rather than guaranteed
- cache hits reported via `prompt_cache_hit_tokens` / `prompt_cache_miss_tokens`
- unused cache entries normally cleared after **a few hours to a few days**

Official docs:

- https://api-docs.deepseek.com/guides/kv_cache
- https://api-docs.deepseek.com/api/create-chat-completion

There is no provider contract corresponding to a client-visible deterministic `TTL = 300s`.

## Likely code path

The direct DeepSeek model uses the OpenAI-compatible / `openai-completions` lane. The generic prompt-cache TTL resolver appears to classify that lane as a 300-second cache lifetime. #767 then consumes the resolved safe-wait budget and schedules the Goal wake at 270 seconds.

Conceptually:

```text
DeepSeek official API
-> openai-completions
-> resolvePromptCacheTtlSeconds() ~= 300
-> safety buffer 30s
-> safe wait 270s
-> Goal monitor wake every 4m30
```

This also causes the TUI and cache-savings estimator to make a provider-specific claim ("5m prompt-cache TTL") that DeepSeek itself does not make.

## Expected behavior

Senpi should distinguish providers with an explicit/deterministic cache TTL from providers with automatic best-effort caching and no fixed TTL contract.

For direct DeepSeek specifically, it should not:

1. report a fabricated 5-minute prompt-cache TTL;
2. wake the Goal every 270 seconds solely to preserve that assumed TTL;
3. estimate cache-warm savings under the assumption that a 270s wake is necessary to keep the DeepSeek cache alive.

## Important edge case

Simply changing the DeepSeek resolver result from `300` to `undefined` may not be sufficient: #767 intentionally maps an unknown cache budget to the legacy 240-second fallback. That would change 4m30 wakes into 4m wakes, not solve the underlying issue.

The missing semantic may be closer to something like:

```text
explicit TTL cache -> use TTL-aware safe-wait scheduling
automatic/best-effort -> do not schedule a wake solely for cache preservation
unknown cache semantics -> existing fallback policy, if desired
```

I would avoid hard-coding DeepSeek to `3600`, because DeepSeek documents "a few hours to a few days", not a guaranteed one-hour TTL.

## Related work

This seems closely related to the recent cache-hit correctness work:

- #766 — provider cache correctness / TTL reporting fixes
- #767 — derive Goal continuation timing from prompt-cache safe-wait budget
- #820 — align Anthropic TTL resolver with the provider's actual cache-retention contract

The same principle from #820 seems applicable here: the runtime TTL estimate used by cache-aware scheduling should reflect the provider's real cache semantics rather than a generic OpenAI-compatible default.

## Reproduction

1. Use Senpi/OmO with the official DeepSeek API provider/model.
2. Run a Goal with a live wake source/monitor so the Goal continuation is deferred.
3. Observe the cache-warm notice.
4. The scheduled delay is 4m30 and the notice reports a 5m prompt-cache TTL.

No provider failure or retry is required; this is steady-state Goal scheduling behavior.

Contributor guide

Open the contributing guide

Research direction

Start at resolvePromptCacheTtlSeconds and the Goal monitor continuation path described in #767; compare the related cache work in #766 and #820. Reproduce with a direct DeepSeek Goal and inspect the TUI wording and cache-savings estimate. Done means best-effort caching is not presented as a fixed TTL or used alone to schedule periodic wakes, while explicit TTL behavior and the existing unknown-budget policy remain distinguishable.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.