elastic / elastic/integrations

[aws_bedrock] ECS GenAI Field Support for detections

Open
#20,738 2 comments 0 reactions 0 assignees View on GitHub
Integration:aws_bedrock Team:Security-Service Integrations
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

**Related issues:**
- https://github.com/elastic/integrations/issues/20737
- https://github.com/elastic/integrations/issues/20644

TRaDE is requesting that the following fields be added to this integration for security detections:

## Requested fields
| Field | Availability | Status |
|---|---|---|
| `gen_ai.provider.name` | ✅ Inferable from `model_id` prefix (e.g., `anthropic.*` → `"aws_bedrock"`) | ❌ Not implemented — integration uses non-standard `gen_ai.system = "aws"` instead |
| `gen_ai.request.model` | ✅ `model_id` present in source logs | ⚠️ Incorrect naming — collected as `gen_ai.request.model.id` (treats model as object); ECS defines this as a flat keyword |
| `gen_ai.response.model` | ⚠️ Not included in Bedrock response payloads; same as request model in all standard APIs | ❌ Field defined in schema but not populated by pipeline |
| `gen_ai.request.stop_sequences` | ✅ `stop_sequences` present in `input_body_json` for text models | ❌ Not implemented — pipeline extracts `max_tokens`/`temperature`/`top_k`/`top_p` but not `stop_sequences` |
| `gen_ai.operation.name` | ✅ `operation` field present in source (e.g., `InvokeModel`, `Converse`, `InvokeModelWithResponseStream`) | ⚠️ Incorrect mapping — `operation` is copied to `event.action` only; `gen_ai.operation.name` is never set |
| `gen_ai.usage.input_tokens` | ✅ `input_token_count` present in source logs | ⚠️ Incorrect naming — collected as `gen_ai.usage.prompt_tokens`; ECS standard is `gen_ai.usage.input_tokens` |
| `gen_ai.usage.output_tokens` | ✅ `output_token_count` present in source logs | ⚠️ Incorrect naming — collected as `gen_ai.usage.completion_tokens`; ECS standard is `gen_ai.usage.output_tokens` |
| `gen_ai.input.messages` | ✅ `messages[]` present in `input_body_json` and partially normalized in pipeline | ⚠️ Incorrect naming — messages stored as `aws_bedrock.invocation.messages` and raw JSON in `gen_ai.prompt`; `gen_ai.input.messages` (ECS flattened) not populated |
| `gen_ai.output.type` | ⚠️ Inferable from `task_type` field (e.g., `TEXT`, `EMBEDDING`, `IMAGE`) but not a direct log field | ❌ Not implemented |
| `gen_ai.output.messages` | ✅ Response content present in `output_body_json` | ❌ Not implemented — raw response stored in `gen_ai.completion` (serialized JSON string); `gen_ai.output.messages` (ECS structured) not populated |
| `gen_ai.system_instructions` | ✅ System prompt present in `input_body_json.system` for Converse API | ⚠️ Incorrect naming — stored as `gen_ai.request.model.instructions` (non-standard); `gen_ai.system_instructions` (ECS flattened) not populated |
| `gen_ai.tool.definitions` | ✅ Available in `input_body_json.tools` when tools are configured in the request | ❌ Not implemented |
| `gen_ai.tool.call.id` | ✅ Present in `tool_use` content blocks in output messages | ❌ Not implemented |
| `gen_ai.tool.name` | ✅ Present in `tool_use` and `tool_result` content blocks | ❌ Not implemented — `gen_ai.analysis.tool_names` exists but is unrelated (security analysis context) |
| `gen_ai.tool.type` | ✅ Not a discrete field in Bedrock logs; type is always `function` for standard tools and can be inferred | ❌ Not implemented |
| `gen_ai.tool.call.result` | ✅ Present in `tool_result` content blocks in input messages | ❌ Not implemented |
| `gen_ai.tool.call.arguments` | ✅ Present in `tool_use.input` blocks in output messages | ❌ Not implemented |

## Likely already implemented (verify on development)
| Field | Availability | Status |
|---|---|---|
| `gen_ai.request.max_tokens` | ✅ `max_tokens` present in `input_body_json` | ✅ Implemented |
| `gen_ai.request.temperature` | ✅ `temperature` present in `input_body_json` | ✅ Implemented |
| `gen_ai.response.finish_reasons` | ✅ `stop_reason` present in output body | ✅ Implemented (mapped from `stop_reason`) |
| `gen_ai.response.id` | ✅ `output_body_json.id` present for Claude/Converse models | ✅ Implemented |

## Likely not possible (verify on development)
| Field | Availability | Status |
|---|---|---|
| `gen_ai.request.seed` | ❌ Not a parameter in standard text model invocations; only exists in `image_generation_config.seed` | ❌ Not implemented |
| `gen_ai.conversation.id` (OTel) | ❌ Not available in standard `InvokeModel`/`Converse` logs; Bedrock Agents sessions have IDs but require new datastream | ❌ Not implemented |
| `gen_ai.token.type` | ❌ Not a discrete field in Bedrock logs; token counts are aggregated per invocation | ❌ Not implemented |

## Thinking Block Collection — End-to-End Configuration Guide

**Provider config:**
- Enable model invocation logging via console or `PutModelInvocationLoggingConfiguration`
- Caller must use extended thinking on the request (`"thinking": {"type": "enabled", "budget_tokens": N}`)
- Set `display: "summarized"` (not omitted) to get readable content
- For responses >100 KB, configure S3 destination or content is truncated

**What lands in raw data:**
```json
output.outputBodyJson.content[]: [
{"type": "thinking", "thinking": "...", "signature": "..."},
{"type": "text", "text": "..."}
]
```
Also appears in `messages[]` in the input body when thinking blocks are passed as prior assistant turn context.

**Integration gap:** No field or pipeline processor extracts thinking blocks from `outputBodyJson.content[]`. Would need a new field (e.g. `gen_ai.output.messages` once schema supports it, or an integration-specific field) and a pipeline processor that filters `content[]` by `type == "thinking"`.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the aws_bedrock integration pipeline and ECS mappings for input_body_json and output.outputBodyJson.content[]. Verify the listed fields against development data, including thinking blocks and fields marked likely implemented or not possible. Done means the feasible ECS fields are mapped, unsupported fields are confirmed, and the resulting structured data is validated.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.