agentscope-ai / agentscope-ai/QwenPaw

[Feature] DeepSeek models: add native capability metadata, prompt-prefix stability, and KV-cache observability (informed by deepseek-harness design)

Ouverte
#7,717 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
enhancement
Langage dominant
Python
Étoiles
34.9k
Forks
3.1k
Merge moyen
1 j 15 h
PR mergées (30 j)
225

Description

Summary
What: This proposal asks for four provider-level enhancements for DeepSeek models in QwenPaw. They are inspired by the design of DeepSeek's official harness (deepseek-ai/deepseek-harness, "dsh"), studied against QwenPaw 2.2.0's current implementation.
Why: DeepSeek's V4.1 Flash (API id deepseek-flash, released 2026-09-10) is architecturally optimized around prompt/KV caching — its own model card frames the release around KV-cache compression, and DeepSeek's API pricing makes a cache hit ≈50× cheaper than a miss (off-peak: ¥0.02 vs ¥1.00 per M input tokens). Third-party benchmarks also report that during real agent sessions on DeepSeek's API, cache hit rates in the 97–99% range are achievable, and that harnesses differ mainly on cost-efficiency rather than raw task solving. As more QwenPaw users route to DeepSeek models (including deepseek-flash), prefix stability and cache visibility become user-visible cost/UX levers, not micro-optimizations.
Scope: All four items are additive and opt-in-friendly. Item 1 is a pure metadata change. Items 2–4 touch provider/agent layers but are designed to be no-op for non-DeepSeek and non-cache-aware models.

Background / Evidence(背景与证据)
Sources consulted:
● deepseek-ai/deepseek-harness @ master (c291e7961a515f6d7af9304e7fd1d257929aef26, 2026-09-10). Key docs: docs/deepseek-llm-api-wire-extensions.md, docs/subsystems/system-prompt.md, packages/llm/llm-deepseek/README.md, .agents/notes/implemented/bug-fix/2026-09-10-deepseek-image-token-calculator-v41.md.
● DeepSeek release notes for V4.1 Flash (2026-09-10) and its pricing table.
● QwenPaw 2.2.0 source as installed (qwenpaw/providers/provider_catalog.py, providers/openai_provider.py, providers/openai_chat_model_compat.py, agents/react_agent.py, agents/middlewares.py, token_usage/manager.py).
● Third-party benchmark reporting on DeepSeek harnesses (Composio harness benchmark, Aug 2026) — treated as directional, not authoritative.
Key observations that motivate each proposal are listed inline below.

Proposal 1 — DeepSeek model capability metadata in the provider catalog ⭐ highest value / smallest diff
Problem. DeepSeek models are currently served through a generic OpenAI-compatible provider:
python
# qwenpaw/providers/provider_catalog.py
PROVIDER_DEEPSEEK = OpenAIProvider(
id="deepseek", name="DeepSeek",
base_url="https://api.deepseek.com",
api_key_prefix="sk-",
models=DEEPSEEK_MODELS,
support_model_discovery=True,
merge_with_catalog=True, freeze_url=True,
)
and provider_discovery_policy.py resolves it as _OPENAI_DYNAMIC. As a result, model-level facts that DeepSeek documents (and that its own harness consumes) have no place to live in QwenPaw: input modalities, context window, thinking/reasoning effort levels, and — for the new deepseek-flash — the system-prompt-update mode.
Requested change. Extend the model metadata schema (wherever DEEPSEEK_MODELS / capability baseline lands) so a model entry can declare, at minimum:
Field Example (deepseek-flash
) Consumer
input_modalities ["text", "image"] skip blind multimodal probing; enable image content parts
context_window 1000000 context management
reasoning.efforts ["off","low","high","max"] thinking controls UI + request params
reasoning.default "high" request defaults
system_prompt_update "in-history"
(see Proposal 2) agent layer prompt strategy
image_pixel_budget
(if images supported) e.g. 640000 request-side image downscale policy
Verification (how to reproduce). In dsh the same facts are declared in packages/llm/llm-deepseek/src/index.ts (DEFAULT_MODELS), e.g. deepseek-flash → name: "DeepSeek-V41-Flash", inputModalities: ["text","image"], systemPromptUpdate: "in-history". In QwenPaw, grep -n "deepseek" qwenpaw/providers/ shows no per-model facts of this kind.
Acceptance criteria.
1. A deepseek-flash entry declares image input; a chat with an attached image no longer falls back to text-only behavior.
2. Optional: documented in website/public/docs/models.*.md.

Proposal 2 — Prompt-prefix stability: support appending changed system prompt after cached history ("in-history")
Problem. QwenPaw builds one system prompt per agent construction (agents/react_agent.py passes system_prompt=system_prompt into the AgentScope Agent; agents/prompt_builder.py assembles host anchors + plugin sections). AgentScope sends it as the leading system message. Any change to that leading content — or any provider-side prompt caching that keys off it — invalidates reuse from the first changed token, i.e. the entire prefix must be re-prefilled. On long, tool-heavy sessions this is the dominant cost term for DeepSeek models, whose API bills cache misses ≈50× cache hits.
Requested change. Two coordinated steps:
1. Carrier: expose an optional per-model system_prompt_update capability (Proposal 1). Value "in-history" means: the endpoint reads the latest system message at any position of messages as the complete effective system prompt (this is DeepSeek's documented semantic, used by dsh).
2. Strategy: when the capability is present and the session series is continuing, a changed non-empty system prompt may be appended as a later system-role message after the cached history instead of rewriting the leading message. When absent, behavior stays exactly as today (rewrite/consolidate at the head).
Why this is safe. It is strictly a transport-position choice for a model family that reads system messages position-independently. Non-supporting models never see the new path. Content/semantics preserved: the model still receives the complete latest prompt.
Verification. dsh implements exactly this and documents the rationale in docs/subsystems/system-prompt.md and packages/core/agent-loop/README.md (search: systemPromptUpdate / in-history). Their stated benefit: "a system prompt change inside a continuing request series is appended after the cached history, so the prefix through that history stays reusable."
Acceptance criteria.
1. With the capability set, a mid-session prompt change produces an append (verify request body / provider logs), and a cache-hit rate measurement (Proposal 4) does not collapse after the change.
2. With the capability unset, byte-for-byte identical request layout to current behavior (regression test).

Proposal 3 — Stabilize tool-schema ordering within a session
Problem. Tool schemas are part of the model-visible prefix. If the assembled tool list changes order between turns without a semantic change (set iteration order, registration order, plugin load timing), the cache breaks from the first reordered schema onward. dsh treats this as a first-class concern: it records the "authoritative returned tool order" (toolOrder, or lexicographic when unset) in a per-request header snapshot, and freezes the header for unchanged envelopes.
Requested change. Freeze the serialized tool-schema order for the life of a session (or until the tool set itself changes), e.g. sort deterministically by tool name at request-build time. Optionally log the order once per session for debuggability.
Verification. In QwenPaw, check the request-building path where toolkit schemas are serialized; run one session, capture two consecutive request bodies, diff the tools array ordering. If it can differ with unchanged tool set, this proposal applies.
Acceptance criteria. Consecutive requests within a session with an unchanged tool set are byte-identical in the tools array.

Proposal 4 — KV-cache observability (surface hit rate + alert threshold)
Problem. QwenPaw already records cache reads: token_usage/manager.py defines cache_read_tokens / cache_hit_rate. But nothing computes a session-level hit-rate trend, and nothing warns when prefix stability degrades. Users on DeepSeek pay 50× more for misses and cannot see why.
Requested change.
1. Surface per-session/per-day cache hit rate in the existing usage surfaces (where token usage is already shown/logged).
2. Optional low-severity notice when a long session's hit rate falls below a configurable threshold (e.g. < 70%) — "prefix stability degraded; check recent prompt/tool changes".
Acceptance criteria. A long local session shows a hit-rate number derived from provider usage fields; threshold notice fires in a synthetic degraded case.

Non-goals / Notes
● These proposals deliberately do not ask QwenPaw to reimplement dsh's plugin architecture, session-log upload extensions, or any DeepSeek-specific telemetry. Only the four portable, user-visible improvements above are requested.
● Proposal 1 is a strict prerequisite for Proposal 2; 3 and 4 are independent.
● If maintainers prefer, 3 and 4 could be narrowed to "DeepSeek models only" to reduce blast radius.
● Related prior art worth one line in the issue (if reviewers ask): Anthropic/OpenAI prompt-caching docs describe similar prefix-stability economics; DeepSeek's public KV-cache guide documents the pricing mechanism used in this proposal.

Checklist for submitter (not part of issue text)
● Searched existing issues for duplicates (cache / prefix / deepseek)
● Confirmed the repository issue template/format requirements
● Attached no logs containing API keys
● Ready to engage as proposer (respond to maintainer questions; PR can follow CONTRIBUTING.md gates: pre-commit run --all-files + pytest)

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.