aws / aws/bedrock-agentcore-sdk-python
[FEATURE] AgentCore Harness: expose prompt caching (Strands cache_config) as a Harness-level config option
- Ngôn ngữ chính
- Python
- Star
- 764
- Fork
- 148
- Merge trung bình
- 1 ngày 23 giờ
- Pull request đã merge (30 ngày)
- 7
Mô tả
**Is your feature request related to a problem? Please describe.**
There's no way to enable Bedrock prompt caching on an AgentCore Harness today. The underlying Strands Agents framework supports it — `BedrockModel(cache_config=CacheConfig(strategy="auto"))` emits `cachePoint` blocks into the Converse request — and Bedrock itself supports per-content-block cache checkpoints. But the Harness API's `HarnessBedrockModelConfig.additionalParams` is a **passthrough to Bedrock Converse request fields**, NOT to Strands framework kwargs, so there's no user-controllable path to the cache_config knob.
Empirical repro from a Console Harness playground call:
```bash
aws bedrock-agentcore-control update-harness \
--model 'bedrockModelConfig={
modelId=global.anthropic.claude-sonnet-4-6,
additionalParams={cache_config={strategy=auto}}
}'
```
The `additionalParams` value ends up as a top-level field on Bedrock's Converse API request, which rejects it:
```
Parameter validation failed:
Unknown parameter in input: "cache_config",
must be one of: modelId, messages, system, inferenceConfig, toolConfig,
guardrailConfig, additionalModelRequestFields, promptVariables,
additionalModelResponseFieldPaths, requestMetadata, performanceConfig,
serviceTier, outputConfig
```
That error confirms two things:
1. `additionalParams` maps 1:1 to Bedrock Converse request fields.
2. Cache configuration doesn't exist at the Converse level — it's per-content-block (`cachePoint` markers inside `system`/`messages`/`tools`), which the Strands framework emits *before* firing Bedrock. Without a way to talk to the framework, caching stays off.
**Cost impact.** For our ADAPT pipeline (multi-stage MEC error correction agent), each invocation runs ~40–75 model turns with a ~150K plateau input-token conversation context, re-billed as fresh input every turn. That's **~$19–25/session** today. Enabling `cache_config="auto"` on the stable ~10K-token system prompt would drop this to ~$5–6 (roughly 75% saving). At production target of ~200 issues/day this is the difference between ~$450/day and ~$1,900/day. Every stable-system-prompt workload on managed Harness pays this cost.
**Describe the solution you'd like**
A Harness-level prompt caching config that the managed Strands runtime picks up and translates into `cachePoint` markers on the Converse request. Two possible shapes:
**Option A — dedicated field (cleaner, no doc contract change):**
```json
{
"bedrockModelConfig": {
"modelId": "global.anthropic.claude-sonnet-4-6",
"promptCaching": {
"strategy": "auto",
"ttl": "5m"
}
}
}
```
**Option B — framework-kwargs passthrough (would require semantic split from today's `additionalParams`, since that's Converse passthrough):**
```json
{
"bedrockModelConfig": {
"modelId": "global.anthropic.claude-sonnet-4-6",
"frameworkParams": {
"cache_config": {"strategy": "auto"}
}
}
}
```
Semantics that would match Strands directly: `strategy` ∈ `{"auto", "anthropic"}`, optional `ttl` (`"5m"` / `"1h"`), and optionally `cache_tools` for tool-schema caching (also supported by Strands' `BedrockModel`).
**Describe alternatives you've considered**
1. **Bring-your-own agent runtime** — skip the managed Harness path and deploy a custom AgentCore Runtime container with self-managed Strands where `cache_config` is set at BedrockModel construction. Big architectural change; loses what Harness gives (managed microVM per session, tool routing, memory config, InvokeHarness single-call API).
2. **Fork Strands and set `cache_config` unconditionally** — the managed Harness runtime pins its own Strands version, so a user fork doesn't apply.
3. **`additionalParams.additionalModelRequestFields.anthropic_beta = ["prompt-caching-2024-07-31"]`** — activates the model-level capability, but the request still needs per-content-block `cache_control` markers that Strands isn't emitting.
4. **AWS support ticket** — contract-level path, no public paper trail; not useful for other customers hitting the same gap.
**Additional context**
- Strands cache support: [`BedrockModel.cache_config` parameter in strands-agents/harness-sdk](https://github.com/strands-agents/harness-sdk/blob/main/strands-py/src/strands/models/bedrock.py)
- Bedrock prompt caching reference: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html
- Terraform provider gap for the same overall class of pass-through knobs: [hashicorp/terraform-provider-aws#48363](https://github.com/hashicorp/terraform-provider-aws/issues/48363)
- Related in this repo: [#648 — `environmentVariables` not injected into microVM process env](https://github.com/aws/bedrock-agentcore-sdk-python/issues/648)
For managed Harness, this is the biggest cost lever available for stable-prompt agent workloads. Every large-context multi-turn agent will re-pay for the same ~10K–50K system prompt every turn until the framework layer gets a way to opt into caching.
Hướng dẫn đóng góp
Hướng nghiên cứu
Bắt đầu với strands-py/src/strands/models/bedrock.py và tham số BedrockModel.cache_config của nó, sau đó lần theo các entry point HarnessBedrockModelConfig và additionalParams được mô tả trong issue. Công việc được xem là hoàn tất khi một tùy chọn được ghi tài liệu ở cấp Harness đi tới runtime Strands được quản lý, tạo ra các marker cachePoint trong các request Converse và hỗ trợ chiến lược cũng như ngữ nghĩa TTL đã nêu.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- aws, python
- Lĩnh vực
- backend-api-design, cloud
- Loại issue
- Tính năng
- Độ khó
- 5/5
- Thời gian dự kiến
- Hơn một tuần
- Mức độ hoạt động
- Sôi nổi
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 45/100