aws / aws/bedrock-agentcore-sdk-python

gen_ai spans (invoke_agent / execute_tool) omit calling user identity and client IP — add enduser.id / client.address

Offen
#577 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
761
Forks
147
Ø Merge
1 T. 23 Std.
Gemergte PRs (30 T.)
7

Beschreibung

## Summary

The GenAI observability spans emitted for an AgentCore-hosted agent
(`invoke_agent`, `execute_tool`, `chat`, `POST /invocations`) contain **no identity
of the caller who invoked the agent and no client IP address**. The invoking IAM
principal / end user and the real client address are absent from every span and from
the correlated GenAI content log records. This makes it impossible to attribute an
agent session (and its tool calls) to *who* triggered it using CloudWatch
observability alone.

## Observed spans — no identity/network attributes

For a full invocation I inspected every span (verified live). None carry any
caller-identity or client-IP attribute:

- `invoke_agent Strands Agents` — has `gen_ai.*`, `gen_ai.usage.*`, `harness.id`,
`session.id`; **no** user or client address.
- `execute_tool ` — has `gen_ai.tool.*`, `session.id`; **no** user or client address.
- `chat` / `chat ` — model attrs only.
- `POST /invocations` (LOCAL_ROOT) — has only **internal** transport values
(`net.peer.ip = 127.0.0.1`, `http.host = 127.0.0.1:8080`,
`http.url = http://cell01.us-east-2.prod.arp.kepler-analytics.aws.dev/invocations`);
the external caller's IP/identity is not present.

So neither the identity nor the true client address of the `InvokeAgentRuntime`
caller is propagated onto the spans.

## Relevant OpenTelemetry semantic conventions (the schema gap)

These spans should be able to carry the caller identity and client address using
existing OTel conventions:

**End user identity** — https://opentelemetry.io/docs/specs/semconv/registry/attributes/enduser/
| Attribute | Type | Description |
|---|---|---|
| `enduser.id` | string | Unique identifier of an end user (username/email/…) — PII |
| `enduser.pseudo.id` | string | Pseudonymous (non-PII) end-user identifier |
| `user.roles` | string[] | (replaces deprecated `enduser.role`) |

**Client / network** — https://opentelemetry.io/docs/specs/semconv/registry/attributes/client/
| Attribute | Stability | Description |
|---|---|---|
| `client.address` | Stable | Client address behind any intermediaries (the real caller IP) |
| `client.port` | Stable | Client port |
| `user_agent.original` | Stable | Client user agent |

None of these are populated on the AgentCore/Strands GenAI spans.

## Request

Stamp the invoking caller's identity and client address onto the agent spans
(at minimum the root `invoke_agent` / `POST /invocations` span, ideally inherited by
child `execute_tool` spans), using the OTel conventions above:

- `enduser.id` (or `enduser.pseudo.id` for privacy) — the invoking principal / end
user (e.g. from the `InvokeAgentRuntime` caller identity, or from the
`X-Amzn-Bedrock-AgentCore-Runtime-User-Id` header when
`InvokeAgentRuntimeForUser` is used).
- `client.address` (+ `client.port`, `user_agent.original`) — the real client that
called `InvokeAgentRuntime`, not the internal `127.0.0.1` loopback currently on
`POST /invocations`.

Gating these behind an opt-in (they are sensitive/PII) is fine.

## Why this can't be worked around today

The caller identity and real client IP **do** exist in the `InvokeAgentRuntime`
**CloudTrail data event** (`userIdentity`, `sourceIPAddress`, `userAgent`) — but that
event has `requestParameters: null` and shares **no** join key with the spans
(`session.id` is not in CloudTrail; `requestID`/`eventID` are not in any span). A full
field-value intersection of the two returns only the region. So identity cannot be
attached to a span/session from telemetry alone — it must be captured client-side. Put
another way: the data exists in CloudTrail but is **unjoinable** to the spans, which is
exactly why stamping `enduser.id` / `client.address` onto the spans is needed.

## Environment

- Amazon Bedrock AgentCore managed harness (Strands agent, `amazon.nova-lite-v1:0`),
region us-east-2.
- Spans from `strands.telemetry.tracer`, `telemetry.auto.version = 0.18.0-aws`,
delivered to CloudWatch `aws/spans` (Transaction Search enabled).
- Verified: no `enduser.*`, `client.address`, or external `sourceIPAddress` on any
span; `POST /invocations` shows only internal loopback transport values.

## Steps to reproduce the behavior

**Prereqs:** an AgentCore-hosted agent (e.g. managed harness running Strands) with at
least one tool, and CloudWatch Transaction Search enabled (spans in `aws/spans`).

1. **Invoke the agent** with a fresh session id, from a client with an identifiable AWS
principal (assumed-role/SSO) and a routable public IP:

```python import boto3, uuid c = boto3.client("bedrock-agentcore", region_name="") sid = "repro-" + uuid.uuid4().hex resp = c.invoke_harness( # or invoke_agent_runtime
harnessArn="",
runtimeSessionId=sid,
messages=[{"role": "user", "content": [{"text":
"Use the tool to get info, then summarize it in one sentence."}]}], ) for _ in resp.get("stream", []):
pass print("session:", sid)

2. Pull the spans for that session from CloudWatch Logs group  aws/spans  and dump
every span's attributes:
limit=5000, filterPattern=f'"{sid}"') for e in r["events"]:
m = json.loads(e["message"])
if m.get("name"):
print(m["name"], "→", sorted(m.get("attributes", {})))
3. Inspect the root span  POST /invocations  and the  invoke_agent  /  execute_tool 
spans for any caller-identity or client-address attribute.

Result: no span contains  enduser.id ,  enduser.pseudo.id ,  client.address ,
 client.port , or  user_agent.original . The only network values are internal
( POST /invocations :  net.peer.ip = 127.0.0.1 ,  http.host = 127.0.0.1:8080 ); the
external caller's IP and identity never appear. (The real caller IP + IAM principal are
present only in the  InvokeAgentRuntime  CloudTrail data event, which shares no join
key with these spans.)

Expected behavior

The invoking caller's identity and real client address should be recorded on the agent
spans using the standard OpenTelemetry conventions, so a session/trace can be attributed
to who triggered it from CloudWatch alone:

•  enduser.id  (or  enduser.pseudo.id  for a non-PII identifier) = the invoking
principal / end user (from the  InvokeAgentRuntime  caller identity, or from
 X-Amzn-Bedrock-AgentCore-Runtime-User-Id  when  InvokeAgentRuntimeForUser  is used).
•  client.address  (+  client.port ,  user_agent.original ) = the real client that
called  InvokeAgentRuntime  — not the internal  127.0.0.1  loopback.

At minimum these should appear on the root  invoke_agent  /  POST /invocations  span and
be inheritable by child  execute_tool  spans. Gating them behind an opt-in (PII) is fine.

Additional context

• This is a service-level gap: the agent runs behind the AgentCore runtime's internal
proxy, so the agent/tracer only sees  127.0.0.1 . Only the managed runtime/service can
inject the true external caller identity/IP — hence filed here rather than against the
Strands SDK.
• Cannot be worked around from telemetry: the caller identity + real IP exist in the
 InvokeAgentRuntime  CloudTrail data event ( userIdentity ,  sourceIPAddress ,
 userAgent ), but that event has  requestParameters: null  and shares no per-invocation
key with the spans — a full field-value intersection of the two returns only the region.
(Companion request: "Add  runtimeSessionId  to the  InvokeAgentRuntime  CloudTrail data
event" — the two together close the attribution gap from either side.)
• OTel references: end-user attrs
https://opentelemetry.io/docs/specs/semconv/registry/attributes/enduser/ ·
client attrs https://opentelemetry.io/docs/specs/semconv/registry/attributes/client/
• Env: managed harness, Strands agent ( amazon.nova-lite-v1:0 ), region us-east-2;
spans from  strands.telemetry.tracer ,  telemetry.auto.version = 0.18.0-aws .

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne damit, die Telemetrie-Einstiegspunkte des SDK rund um strands.telemetry.tracer und die Spans des verwalteten Harness nachzuverfolgen, und prüfe anschließend, ob die Identität des Aufrufers und die Client-Adresse für dieses Repository verfügbar sind oder nur für den AgentCore-Service. Vergleiche die Spans von POST /invocations, invoke_agent und execute_tool in CloudWatch und berücksichtige die OpenTelemetry-Konventionen für enduser und client; als abgeschlossen gilt die Aufgabe, wenn ein unterstütztes Design oder eine Implementierung Spans zuordnen kann, ohne auf nicht verknüpfbare CloudTrail-Daten angewiesen zu sein.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
aws, python
Bereich
backend, observability
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.