aws / aws/bedrock-agentcore-sdk-python

OTEL context token detached across asyncio/thread boundary in memory client + Strands session_manager

オープン
#456 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
761
フォーク
147
平均マージ
1日 23時間
マージ済み PR(30日)
7

説明

## Summary

When using `bedrock-agentcore` (1.6.2) with the Strands integration inside an AgentCore runtime, the OpenTelemetry runtime logs a steady stream of `Failed to detach context` errors:

```
ERROR [opentelemetry.context] [__init__.py:157] - Failed to detach context
Traceback (most recent call last):
File "/var/task/opentelemetry/context/__init__.py", line 155, in detach
_RUNTIME_CONTEXT.detach(token)
File "/var/task/opentelemetry/context/contextvars_context.py", line 53, in detach
self._current_context.reset(token)
ValueError: at 0x...> was created in a different Context
```

The errors are logged from spans owned by:

- `bedrock_agentcore.memory.client` (around `create_event` / `client.py:484`)
- `bedrock_agentcore.memory.integrations.strands.session_manager` (`session_manager.py:406`, around `Created agent: default in session: …`)

Functionally tracing still works (events are created, agents start), but ERROR-level log spam pollutes CloudWatch and obscures real failures.

## Root cause (best read)

The SDK calls `opentelemetry.context.attach(ctx)` to set span context, then `opentelemetry.context.detach(token)` to restore. Per PEP 567, every `asyncio.Task` (and every threadpool worker) gets its own copy of the contextvars state, so a token created in one execution context is not valid to `reset()` in another.

This shows up specifically when:

1. A boto3 sync call (e.g., `CreateEvent`) is invoked from inside an `async` method and ends up resuming on a different thread / task than the one that called `attach`.
2. A Strands session manager hook (e.g. `Created agent` event) attaches in one task and detaches after an `await` resumes on another task.

## Repro environment

- `bedrock-agentcore==1.6.2`
- `bedrock-agentcore-starter-toolkit==0.3.5`
- AgentCore-managed runtime (Lambda-style), `aws-opentelemetry-distro` enabled
- Strands `Agent` constructed with `AgentCoreMemorySessionManager`
- Concurrent module-builder agents launched from a `ThreadPoolExecutor`

## Expected

`detach()` should not raise (or the SDK should not call it across context boundaries). Either:

- Use `with use_span(...)` / `set_value` patterns that don't require explicit `detach`, or
- Capture the originating context and only detach if the current context still owns the token, or
- Wrap the detach in a try/except `ValueError` since a stale token can be safely ignored.

## Workaround consumers can apply

```python
from opentelemetry import context as _ctx
_orig_detach = _ctx.detach
def _safe_detach(token):
try:
_orig_detach(token)
except ValueError:
pass
_ctx.detach = _safe_detach
```

…installed before importing `bedrock_agentcore`. This is purely cosmetic and should live in the SDK instead.

## Suggested fix

In `bedrock_agentcore.memory.client` and `bedrock_agentcore.memory.integrations.strands.session_manager`, wrap each `context.detach(token)` call with a `try/except ValueError` (or refactor to use `opentelemetry.trace.use_span` so no manual token management is needed across `await` boundaries).

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

bedrock_agentcore/memory/client.py の create_event と client.py:484 付近から開始し、次に bedrock_agentcore/memory/integrations/strands/session_manager.py の 406 行目付近を調べます。async とスレッドの境界をまたぐ各 OpenTelemetry の attach/detach ペアを追跡し、報告されたランタイムログを再現します。memory と Strands のフローで、Failed to detach context エラーなしにトレーシングが維持されれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
observability
issue の種類
バグ
難易度
3/5
見積もり時間
1〜2日
活発さ
静か
明瞭さ
明確に書かれている
初心者へのやさしさ
68/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。