aws / aws/bedrock-agentcore-sdk-python

gen_ai spans (invoke_agent / execute_tool) omit calling user identity and client IP — add enduser.id / client.address

Aperta
#577 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
761
Fork
147
Merge medio
1g 23h
PR unite (30g)
7

Descrizione

## Summary

The GenAI observability spans emitted for an AgentCore-hosted agent
(`invoke_agent`, `execute_tool`, `chat`, `POST /invocations`) contain **no identity
of the caller who invoked the agent and no client IP address**. The invoking IAM
principal / end user and the real client address are absent from every span and from
the correlated GenAI content log records. This makes it impossible to attribute an
agent session (and its tool calls) to *who* triggered it using CloudWatch
observability alone.

## Observed spans — no identity/network attributes

For a full invocation I inspected every span (verified live). None carry any
caller-identity or client-IP attribute:

- `invoke_agent Strands Agents` — has `gen_ai.*`, `gen_ai.usage.*`, `harness.id`,
`session.id`; **no** user or client address.
- `execute_tool ` — has `gen_ai.tool.*`, `session.id`; **no** user or client address.
- `chat` / `chat ` — model attrs only.
- `POST /invocations` (LOCAL_ROOT) — has only **internal** transport values
(`net.peer.ip = 127.0.0.1`, `http.host = 127.0.0.1:8080`,
`http.url = http://cell01.us-east-2.prod.arp.kepler-analytics.aws.dev/invocations`);
the external caller's IP/identity is not present.

So neither the identity nor the true client address of the `InvokeAgentRuntime`
caller is propagated onto the spans.

## Relevant OpenTelemetry semantic conventions (the schema gap)

These spans should be able to carry the caller identity and client address using
existing OTel conventions:

**End user identity** — https://opentelemetry.io/docs/specs/semconv/registry/attributes/enduser/
| Attribute | Type | Description |
|---|---|---|
| `enduser.id` | string | Unique identifier of an end user (username/email/…) — PII |
| `enduser.pseudo.id` | string | Pseudonymous (non-PII) end-user identifier |
| `user.roles` | string[] | (replaces deprecated `enduser.role`) |

**Client / network** — https://opentelemetry.io/docs/specs/semconv/registry/attributes/client/
| Attribute | Stability | Description |
|---|---|---|
| `client.address` | Stable | Client address behind any intermediaries (the real caller IP) |
| `client.port` | Stable | Client port |
| `user_agent.original` | Stable | Client user agent |

None of these are populated on the AgentCore/Strands GenAI spans.

## Request

Stamp the invoking caller's identity and client address onto the agent spans
(at minimum the root `invoke_agent` / `POST /invocations` span, ideally inherited by
child `execute_tool` spans), using the OTel conventions above:

- `enduser.id` (or `enduser.pseudo.id` for privacy) — the invoking principal / end
user (e.g. from the `InvokeAgentRuntime` caller identity, or from the
`X-Amzn-Bedrock-AgentCore-Runtime-User-Id` header when
`InvokeAgentRuntimeForUser` is used).
- `client.address` (+ `client.port`, `user_agent.original`) — the real client that
called `InvokeAgentRuntime`, not the internal `127.0.0.1` loopback currently on
`POST /invocations`.

Gating these behind an opt-in (they are sensitive/PII) is fine.

## Why this can't be worked around today

The caller identity and real client IP **do** exist in the `InvokeAgentRuntime`
**CloudTrail data event** (`userIdentity`, `sourceIPAddress`, `userAgent`) — but that
event has `requestParameters: null` and shares **no** join key with the spans
(`session.id` is not in CloudTrail; `requestID`/`eventID` are not in any span). A full
field-value intersection of the two returns only the region. So identity cannot be
attached to a span/session from telemetry alone — it must be captured client-side. Put
another way: the data exists in CloudTrail but is **unjoinable** to the spans, which is
exactly why stamping `enduser.id` / `client.address` onto the spans is needed.

## Environment

- Amazon Bedrock AgentCore managed harness (Strands agent, `amazon.nova-lite-v1:0`),
region us-east-2.
- Spans from `strands.telemetry.tracer`, `telemetry.auto.version = 0.18.0-aws`,
delivered to CloudWatch `aws/spans` (Transaction Search enabled).
- Verified: no `enduser.*`, `client.address`, or external `sourceIPAddress` on any
span; `POST /invocations` shows only internal loopback transport values.

## Steps to reproduce the behavior

**Prereqs:** an AgentCore-hosted agent (e.g. managed harness running Strands) with at
least one tool, and CloudWatch Transaction Search enabled (spans in `aws/spans`).

1. **Invoke the agent** with a fresh session id, from a client with an identifiable AWS
principal (assumed-role/SSO) and a routable public IP:

```python import boto3, uuid c = boto3.client("bedrock-agentcore", region_name="") sid = "repro-" + uuid.uuid4().hex resp = c.invoke_harness( # or invoke_agent_runtime
harnessArn="",
runtimeSessionId=sid,
messages=[{"role": "user", "content": [{"text":
"Use the tool to get info, then summarize it in one sentence."}]}], ) for _ in resp.get("stream", []):
pass print("session:", sid)

2. Pull the spans for that session from CloudWatch Logs group  aws/spans  and dump
every span's attributes:
limit=5000, filterPattern=f'"{sid}"') for e in r["events"]:
m = json.loads(e["message"])
if m.get("name"):
print(m["name"], "→", sorted(m.get("attributes", {})))
3. Inspect the root span  POST /invocations  and the  invoke_agent  /  execute_tool 
spans for any caller-identity or client-address attribute.

Result: no span contains  enduser.id ,  enduser.pseudo.id ,  client.address ,
 client.port , or  user_agent.original . The only network values are internal
( POST /invocations :  net.peer.ip = 127.0.0.1 ,  http.host = 127.0.0.1:8080 ); the
external caller's IP and identity never appear. (The real caller IP + IAM principal are
present only in the  InvokeAgentRuntime  CloudTrail data event, which shares no join
key with these spans.)

Expected behavior

The invoking caller's identity and real client address should be recorded on the agent
spans using the standard OpenTelemetry conventions, so a session/trace can be attributed
to who triggered it from CloudWatch alone:

•  enduser.id  (or  enduser.pseudo.id  for a non-PII identifier) = the invoking
principal / end user (from the  InvokeAgentRuntime  caller identity, or from
 X-Amzn-Bedrock-AgentCore-Runtime-User-Id  when  InvokeAgentRuntimeForUser  is used).
•  client.address  (+  client.port ,  user_agent.original ) = the real client that
called  InvokeAgentRuntime  — not the internal  127.0.0.1  loopback.

At minimum these should appear on the root  invoke_agent  /  POST /invocations  span and
be inheritable by child  execute_tool  spans. Gating them behind an opt-in (PII) is fine.

Additional context

• This is a service-level gap: the agent runs behind the AgentCore runtime's internal
proxy, so the agent/tracer only sees  127.0.0.1 . Only the managed runtime/service can
inject the true external caller identity/IP — hence filed here rather than against the
Strands SDK.
• Cannot be worked around from telemetry: the caller identity + real IP exist in the
 InvokeAgentRuntime  CloudTrail data event ( userIdentity ,  sourceIPAddress ,
 userAgent ), but that event has  requestParameters: null  and shares no per-invocation
key with the spans — a full field-value intersection of the two returns only the region.
(Companion request: "Add  runtimeSessionId  to the  InvokeAgentRuntime  CloudTrail data
event" — the two together close the attribution gap from either side.)
• OTel references: end-user attrs
https://opentelemetry.io/docs/specs/semconv/registry/attributes/enduser/ ·
client attrs https://opentelemetry.io/docs/specs/semconv/registry/attributes/client/
• Env: managed harness, Strands agent ( amazon.nova-lite-v1:0 ), region us-east-2;
spans from  strands.telemetry.tracer ,  telemetry.auto.version = 0.18.0-aws .

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia tracciando i punti di ingresso della telemetria dell'SDK intorno a strands.telemetry.tracer e agli span dell'harness gestito, quindi verifica se l'identità del chiamante e l'indirizzo del client sono disponibili per questo repository o solo per il servizio AgentCore. Confronta gli span di POST /invocations, invoke_agent ed execute_tool in CloudWatch e considera le convenzioni OpenTelemetry per enduser e client; il lavoro è completo quando un design o un'implementazione supportati possono attribuire gli span senza fare affidamento su dati CloudTrail non correlabili.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, python
Ambito
backend, observability
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.