elastic / elastic/integrations
[openai_chatgpt_enterprise] ECS GenAI Field Support for detections
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
**Related issues:**
- https://github.com/elastic/integrations/issues/20737
- https://github.com/elastic/integrations/issues/19399
TRaDE is requesting comprehensive ECS GenAI field support across the `openai_chatgpt_enterprise` integration. Multiple data stream PRs are already in flight (#20205, #20208, #20209, #20201) but the gen_ai field coverage in each is incomplete.
---
## `app_log` data stream (PR #20205)
Captures connector/tool invocations — request and response pairs between ChatGPT Enterprise and MCP or service connectors.
| Field | Source in raw data | Status |
|---|---|---|
| `gen_ai.provider.name` | Hardcode `"openai"` | ✅ Implemented |
| `gen_ai.conversation.id` | `conversation_id` | ❌ Not mapped |
| `gen_ai.operation.name` | Hardcode `"invoke_tool"` | ❌ Not mapped |
| `gen_ai.tool.name` | `app_name` (e.g. `"Slack"`, `"gdrive"`) | ❌ Not mapped |
| `gen_ai.tool.type` | `app_type`: `"MCP"` → `"extension"`, `"SERVICE"` → `"function"` | ❌ Not mapped |
| `gen_ai.tool.call.arguments` | `input` object (request events) | ❌ Not mapped |
| `gen_ai.tool.call.result` | `output[].response.result` (response events) | ❌ Not mapped |
---
## `conversation_message` data stream (PR #20208)
Captures individual user and assistant messages in ChatGPT Enterprise conversations. Message text is present in `message.content.value`.
| Field | Source in raw data | Status |
|---|---|---|
| `gen_ai.provider.name` | Hardcode `"openai"` | ⚠️ **Bug** — PR sets `gen_ai.system = "openai"` which is not a valid ECS field |
| `gen_ai.response.model` | `message.author.model` (assistant messages) | ✅ Implemented |
| `gen_ai.conversation.id` | `conversation.id` | ❌ Not mapped |
| `gen_ai.operation.name` | Hardcode `"chat"` | ❌ Not mapped |
| `gen_ai.input.messages` | `message.content.value` where `message.author.type == "user"` | ❌ Not mapped |
| `gen_ai.output.messages` | `message.content.value` where `message.author.type == "assistant"` | ❌ Not mapped |
| `gen_ai.response.id` | `message.id` (assistant messages) | ❌ Not mapped |
---
## `custom_agents_log` data stream (PR #20209)
Captures agent lifecycle events (AGENT_CREATED, AGENT_RUN_CREATED) and connector call events (CONNECTOR_CALL_REQUESTED, CONNECTOR_CALL_COMPLETED).
| Field | Source in raw data | Status |
|---|---|---|
| `gen_ai.agent.name` | `event_details.agent_fields.name` | ✅ Implemented |
| `gen_ai.agent.id` | `event_details.agent_id` | ✅ Implemented |
| `gen_ai.tool.name` | `event_details.connector_name` (CONNECTOR_CALL events) | ✅ Implemented |
| `gen_ai.request.model` | `event_details.agent_fields.model_spec.name` | ✅ Implemented |
| `gen_ai.provider.name` | Hardcode `"openai"` | ❌ Not mapped |
| `gen_ai.operation.name` | Derivable from `event_type`: `AGENT_RUN_CREATED` → `"chat"`, `CONNECTOR_CALL_*` → `"invoke_tool"` | ❌ Not mapped |
| `gen_ai.tool.call.id` | `event_details.tool_call_id` (CONNECTOR_CALL events) | ❌ Not mapped |
| `gen_ai.tool.type` | Inferable as `"function"` for connector calls | ❌ Not mapped |
| `gen_ai.conversation.id` | `event_details.conversation_id` | ❌ Not mapped |
---
## `codex_log` data stream (PR #20201)
Captures Codex task events including model usage and tool calls.
| Field | Source in raw data | Status |
|---|---|---|
| `gen_ai.request.model` | `event_details.model` | ✅ Implemented |
| `gen_ai.usage.input_tokens` | `event_details.token_usage.input_tokens` | ✅ Implemented |
| `gen_ai.usage.output_tokens` | `event_details.token_usage.output_tokens` | ✅ Implemented |
| `gen_ai.tool.name` | `event_details.tool` (tool_call events) | ✅ Implemented |
| `gen_ai.provider.name` | Hardcode `"openai"` | ❌ Not mapped |
| `gen_ai.operation.name` | Derivable from `event_details.action` | ❌ Not mapped |
| `gen_ai.tool.call.id` | `event_details.tool_call_id` (if present) | ❌ Not mapped |
---
## Fix
**`gen_ai.system` in `conversation_message` should be corrected to `gen_ai.provider.name`
Contributor guide
Research direction
Review the app_log, conversation_message, custom_agents_log, and codex_log data streams through PRs #20205, #20208, #20209, and #20201. Compare each raw field with the ECS GenAI mappings listed here, including correcting gen_ai.system in conversation_message. Done means the requested provider, conversation, operation, tool, message, response, agent, and usage fields are mapped consistently across the four streams.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- handlebars
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100