aws / aws/bedrock-agentcore-sdk-python
[FEATURE] AgentCore Harness: expose prompt caching (Strands cache_config) as a Harness-level config option
- Lingua principale
- Python
- Stelle
- 761
- Fork
- 147
- Merge medio
- 1g 23h
- PR unite (30g)
- 7
Descrizione
**Is your feature request related to a problem? Please describe.**
There's no way to enable Bedrock prompt caching on an AgentCore Harness today. The underlying Strands Agents framework supports it — `BedrockModel(cache_config=CacheConfig(strategy="auto"))` emits `cachePoint` blocks into the Converse request — and Bedrock itself supports per-content-block cache checkpoints. But the Harness API's `HarnessBedrockModelConfig.additionalParams` is a **passthrough to Bedrock Converse request fields**, NOT to Strands framework kwargs, so there's no user-controllable path to the cache_config knob.
Empirical repro from a Console Harness playground call:
```bash
aws bedrock-agentcore-control update-harness \
--model 'bedrockModelConfig={
modelId=global.anthropic.claude-sonnet-4-6,
additionalParams={cache_config={strategy=auto}}
}'
```
The `additionalParams` value ends up as a top-level field on Bedrock's Converse API request, which rejects it:
```
Parameter validation failed:
Unknown parameter in input: "cache_config",
must be one of: modelId, messages, system, inferenceConfig, toolConfig,
guardrailConfig, additionalModelRequestFields, promptVariables,
additionalModelResponseFieldPaths, requestMetadata, performanceConfig,
serviceTier, outputConfig
```
That error confirms two things:
1. `additionalParams` maps 1:1 to Bedrock Converse request fields.
2. Cache configuration doesn't exist at the Converse level — it's per-content-block (`cachePoint` markers inside `system`/`messages`/`tools`), which the Strands framework emits *before* firing Bedrock. Without a way to talk to the framework, caching stays off.
**Cost impact.** For our ADAPT pipeline (multi-stage MEC error correction agent), each invocation runs ~40–75 model turns with a ~150K plateau input-token conversation context, re-billed as fresh input every turn. That's **~$19–25/session** today. Enabling `cache_config="auto"` on the stable ~10K-token system prompt would drop this to ~$5–6 (roughly 75% saving). At production target of ~200 issues/day this is the difference between ~$450/day and ~$1,900/day. Every stable-system-prompt workload on managed Harness pays this cost.
**Describe the solution you'd like**
A Harness-level prompt caching config that the managed Strands runtime picks up and translates into `cachePoint` markers on the Converse request. Two possible shapes:
**Option A — dedicated field (cleaner, no doc contract change):**
```json
{
"bedrockModelConfig": {
"modelId": "global.anthropic.claude-sonnet-4-6",
"promptCaching": {
"strategy": "auto",
"ttl": "5m"
}
}
}
```
**Option B — framework-kwargs passthrough (would require semantic split from today's `additionalParams`, since that's Converse passthrough):**
```json
{
"bedrockModelConfig": {
"modelId": "global.anthropic.claude-sonnet-4-6",
"frameworkParams": {
"cache_config": {"strategy": "auto"}
}
}
}
```
Semantics that would match Strands directly: `strategy` ∈ `{"auto", "anthropic"}`, optional `ttl` (`"5m"` / `"1h"`), and optionally `cache_tools` for tool-schema caching (also supported by Strands' `BedrockModel`).
**Describe alternatives you've considered**
1. **Bring-your-own agent runtime** — skip the managed Harness path and deploy a custom AgentCore Runtime container with self-managed Strands where `cache_config` is set at BedrockModel construction. Big architectural change; loses what Harness gives (managed microVM per session, tool routing, memory config, InvokeHarness single-call API).
2. **Fork Strands and set `cache_config` unconditionally** — the managed Harness runtime pins its own Strands version, so a user fork doesn't apply.
3. **`additionalParams.additionalModelRequestFields.anthropic_beta = ["prompt-caching-2024-07-31"]`** — activates the model-level capability, but the request still needs per-content-block `cache_control` markers that Strands isn't emitting.
4. **AWS support ticket** — contract-level path, no public paper trail; not useful for other customers hitting the same gap.
**Additional context**
- Strands cache support: [`BedrockModel.cache_config` parameter in strands-agents/harness-sdk](https://github.com/strands-agents/harness-sdk/blob/main/strands-py/src/strands/models/bedrock.py)
- Bedrock prompt caching reference: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html
- Terraform provider gap for the same overall class of pass-through knobs: [hashicorp/terraform-provider-aws#48363](https://github.com/hashicorp/terraform-provider-aws/issues/48363)
- Related in this repo: [#648 — `environmentVariables` not injected into microVM process env](https://github.com/aws/bedrock-agentcore-sdk-python/issues/648)
For managed Harness, this is the biggest cost lever available for stable-prompt agent workloads. Every large-context multi-turn agent will re-pay for the same ~10K–50K system prompt every turn until the framework layer gets a way to opt into caching.
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia da strands-py/src/strands/models/bedrock.py e dal relativo parametro BedrockModel.cache_config, quindi segui gli entry point HarnessBedrockModelConfig e additionalParams descritti nell’issue. Il lavoro è completato quando un’opzione documentata a livello Harness raggiunge il runtime Strands gestito, produce marker cachePoint nelle richieste Converse e supporta la strategia e la semantica TTL indicate.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- aws, python
- Ambito
- backend-api-design, cloud
- Tipo di issue
- Funzionalità
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Attiva
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 45/100