aws / aws/bedrock-agentcore-sdk-python

[FEATURE] AgentCore Harness: expose prompt caching (Strands cache_config) as a Harness-level config option

Aperta
#649 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
enhancement
Lingua principale
Python
Stelle
761
Fork
147
Merge medio
1g 23h
PR unite (30g)
7

Descrizione

**Is your feature request related to a problem? Please describe.**

There's no way to enable Bedrock prompt caching on an AgentCore Harness today. The underlying Strands Agents framework supports it — `BedrockModel(cache_config=CacheConfig(strategy="auto"))` emits `cachePoint` blocks into the Converse request — and Bedrock itself supports per-content-block cache checkpoints. But the Harness API's `HarnessBedrockModelConfig.additionalParams` is a **passthrough to Bedrock Converse request fields**, NOT to Strands framework kwargs, so there's no user-controllable path to the cache_config knob.

Empirical repro from a Console Harness playground call:

```bash
aws bedrock-agentcore-control update-harness \
--model 'bedrockModelConfig={
modelId=global.anthropic.claude-sonnet-4-6,
additionalParams={cache_config={strategy=auto}}
}'
```

The `additionalParams` value ends up as a top-level field on Bedrock's Converse API request, which rejects it:

```
Parameter validation failed:
Unknown parameter in input: "cache_config",
must be one of: modelId, messages, system, inferenceConfig, toolConfig,
guardrailConfig, additionalModelRequestFields, promptVariables,
additionalModelResponseFieldPaths, requestMetadata, performanceConfig,
serviceTier, outputConfig
```

That error confirms two things:
1. `additionalParams` maps 1:1 to Bedrock Converse request fields.
2. Cache configuration doesn't exist at the Converse level — it's per-content-block (`cachePoint` markers inside `system`/`messages`/`tools`), which the Strands framework emits *before* firing Bedrock. Without a way to talk to the framework, caching stays off.

**Cost impact.** For our ADAPT pipeline (multi-stage MEC error correction agent), each invocation runs ~40–75 model turns with a ~150K plateau input-token conversation context, re-billed as fresh input every turn. That's **~$19–25/session** today. Enabling `cache_config="auto"` on the stable ~10K-token system prompt would drop this to ~$5–6 (roughly 75% saving). At production target of ~200 issues/day this is the difference between ~$450/day and ~$1,900/day. Every stable-system-prompt workload on managed Harness pays this cost.

**Describe the solution you'd like**

A Harness-level prompt caching config that the managed Strands runtime picks up and translates into `cachePoint` markers on the Converse request. Two possible shapes:

**Option A — dedicated field (cleaner, no doc contract change):**
```json
{
"bedrockModelConfig": {
"modelId": "global.anthropic.claude-sonnet-4-6",
"promptCaching": {
"strategy": "auto",
"ttl": "5m"
}
}
}
```

**Option B — framework-kwargs passthrough (would require semantic split from today's `additionalParams`, since that's Converse passthrough):**
```json
{
"bedrockModelConfig": {
"modelId": "global.anthropic.claude-sonnet-4-6",
"frameworkParams": {
"cache_config": {"strategy": "auto"}
}
}
}
```

Semantics that would match Strands directly: `strategy` ∈ `{"auto", "anthropic"}`, optional `ttl` (`"5m"` / `"1h"`), and optionally `cache_tools` for tool-schema caching (also supported by Strands' `BedrockModel`).

**Describe alternatives you've considered**

1. **Bring-your-own agent runtime** — skip the managed Harness path and deploy a custom AgentCore Runtime container with self-managed Strands where `cache_config` is set at BedrockModel construction. Big architectural change; loses what Harness gives (managed microVM per session, tool routing, memory config, InvokeHarness single-call API).
2. **Fork Strands and set `cache_config` unconditionally** — the managed Harness runtime pins its own Strands version, so a user fork doesn't apply.
3. **`additionalParams.additionalModelRequestFields.anthropic_beta = ["prompt-caching-2024-07-31"]`** — activates the model-level capability, but the request still needs per-content-block `cache_control` markers that Strands isn't emitting.
4. **AWS support ticket** — contract-level path, no public paper trail; not useful for other customers hitting the same gap.

**Additional context**

- Strands cache support: [`BedrockModel.cache_config` parameter in strands-agents/harness-sdk](https://github.com/strands-agents/harness-sdk/blob/main/strands-py/src/strands/models/bedrock.py)
- Bedrock prompt caching reference: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html
- Terraform provider gap for the same overall class of pass-through knobs: [hashicorp/terraform-provider-aws#48363](https://github.com/hashicorp/terraform-provider-aws/issues/48363)
- Related in this repo: [#648 — `environmentVariables` not injected into microVM process env](https://github.com/aws/bedrock-agentcore-sdk-python/issues/648)

For managed Harness, this is the biggest cost lever available for stable-prompt agent workloads. Every large-context multi-turn agent will re-pay for the same ~10K–50K system prompt every turn until the framework layer gets a way to opt into caching.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia da strands-py/src/strands/models/bedrock.py e dal relativo parametro BedrockModel.cache_config, quindi segui gli entry point HarnessBedrockModelConfig e additionalParams descritti nell’issue. Il lavoro è completato quando un’opzione documentata a livello Harness raggiunge il runtime Strands gestito, produce marker cachePoint nelle richieste Converse e supporta la strategia e la semantica TTL indicate.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, python
Ambito
backend-api-design, cloud
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Attiva
Chiarezza
Abbastanza chiara
Idoneità per principianti
45/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.