aws / aws/bedrock-agentcore-sdk-python

OTEL context token detached across asyncio/thread boundary in memory client + Strands session_manager

Aperta
#456 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
761
Fork
147
Merge medio
1g 23h
PR unite (30g)
7

Descrizione

## Summary

When using `bedrock-agentcore` (1.6.2) with the Strands integration inside an AgentCore runtime, the OpenTelemetry runtime logs a steady stream of `Failed to detach context` errors:

```
ERROR [opentelemetry.context] [__init__.py:157] - Failed to detach context
Traceback (most recent call last):
File "/var/task/opentelemetry/context/__init__.py", line 155, in detach
_RUNTIME_CONTEXT.detach(token)
File "/var/task/opentelemetry/context/contextvars_context.py", line 53, in detach
self._current_context.reset(token)
ValueError: at 0x...> was created in a different Context
```

The errors are logged from spans owned by:

- `bedrock_agentcore.memory.client` (around `create_event` / `client.py:484`)
- `bedrock_agentcore.memory.integrations.strands.session_manager` (`session_manager.py:406`, around `Created agent: default in session: …`)

Functionally tracing still works (events are created, agents start), but ERROR-level log spam pollutes CloudWatch and obscures real failures.

## Root cause (best read)

The SDK calls `opentelemetry.context.attach(ctx)` to set span context, then `opentelemetry.context.detach(token)` to restore. Per PEP 567, every `asyncio.Task` (and every threadpool worker) gets its own copy of the contextvars state, so a token created in one execution context is not valid to `reset()` in another.

This shows up specifically when:

1. A boto3 sync call (e.g., `CreateEvent`) is invoked from inside an `async` method and ends up resuming on a different thread / task than the one that called `attach`.
2. A Strands session manager hook (e.g. `Created agent` event) attaches in one task and detaches after an `await` resumes on another task.

## Repro environment

- `bedrock-agentcore==1.6.2`
- `bedrock-agentcore-starter-toolkit==0.3.5`
- AgentCore-managed runtime (Lambda-style), `aws-opentelemetry-distro` enabled
- Strands `Agent` constructed with `AgentCoreMemorySessionManager`
- Concurrent module-builder agents launched from a `ThreadPoolExecutor`

## Expected

`detach()` should not raise (or the SDK should not call it across context boundaries). Either:

- Use `with use_span(...)` / `set_value` patterns that don't require explicit `detach`, or
- Capture the originating context and only detach if the current context still owns the token, or
- Wrap the detach in a try/except `ValueError` since a stale token can be safely ignored.

## Workaround consumers can apply

```python
from opentelemetry import context as _ctx
_orig_detach = _ctx.detach
def _safe_detach(token):
try:
_orig_detach(token)
except ValueError:
pass
_ctx.detach = _safe_detach
```

…installed before importing `bedrock_agentcore`. This is purely cosmetic and should live in the SDK instead.

## Suggested fix

In `bedrock_agentcore.memory.client` and `bedrock_agentcore.memory.integrations.strands.session_manager`, wrap each `context.detach(token)` call with a `try/except ValueError` (or refactor to use `opentelemetry.trace.use_span` so no manual token management is needed across `await` boundaries).

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia in bedrock_agentcore/memory/client.py, intorno a create_event e client.py:484, quindi esamina bedrock_agentcore/memory/integrations/strands/session_manager.py intorno alla riga 406. Traccia ogni coppia attach/detach di OpenTelemetry attraverso i confini tra async e thread e riproduci il logging runtime segnalato. Il lavoro è completato quando i flussi memory e Strands mantengono il tracing senza errori Failed to detach context.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
observability
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Tranquilla
Chiarezza
Specificata chiaramente
Idoneità per principianti
68/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.