Azure / Azure/Azure-Sentinel

Logstash output plugin logs no longer include stream name and request ID

Open
#14,393 2 comments 4 reactions 3 assignees Claimed by @lamiahjoint View on GitHub
logstach
Dominant language
Python
Stars
6.1k
Forks
3.8k
Avg merge
4d 7h
Merged PRs (30d)
125

Description

**Describe the bug**
Prior to v2.x.x of the [microsoft-sentinel-log-analytics-logstash-output-plugin](https://rubygems.org/gems/microsoft-sentinel-log-analytics-logstash-output-plugin) the output would emit a log containing the stream name.

Since the release of v2.x.x until v2.2.1 this type of logging didn't exist, though it now has been re-added but is missing information that was present before including:

- Stream name
- Output ID
- Request ID

**To Reproduce**

To reproduce the old behaviour:

```Dockerfile
FROM docker.elastic.co/logstash/logstash:8.19.16

RUN cd /usr/share/logstash && \
bin/logstash-plugin install --version 1.1.4 microsoft-sentinel-log-analytics-logstash-output-plugin

COPY --chown=logstash:logstash --chmod=750 logstash.conf /usr/share/logstash/pipeline/logstash.conf
```

```ruby
input {
generator {
lines => [
'Hello World'
]
count => 10
}
}

output {
microsoft-sentinel-log-analytics-logstash-output-plugin {
client_app_Id => "${SENTINEL_CLIENT_APP_ID}"
client_app_secret => "${SENTINEL_CLIENT_APP_SECRET}"
data_collection_endpoint => "${SENTINEL_DATA_COLLECTION_ENDPOINT}"
dcr_immutable_id => ""
dcr_stream_name => ""
tenant_id => "${SENTINEL_TENANT_ID}"
}
microsoft-sentinel-log-analytics-logstash-output-plugin {
client_app_Id => "${SENTINEL_CLIENT_APP_ID}"
client_app_secret => "${SENTINEL_CLIENT_APP_SECRET}"
data_collection_endpoint => "${SENTINEL_DATA_COLLECTION_ENDPOINT}"
dcr_immutable_id => ""
dcr_stream_name => ""
tenant_id => "${SENTINEL_TENANT_ID}"
}
}
```

To reproduce the new behaviour:

```Dockerfile
FROM docker.elastic.co/logstash/logstash:8.19.16

RUN cd /usr/share/logstash && \
bin/logstash-plugin install --version 2.2.1 microsoft-sentinel-log-analytics-logstash-output-plugin

COPY --chown=logstash:logstash --chmod=750 logstash.conf /usr/share/logstash/pipeline/logstash.conf
```

```ruby
input {
generator {
lines => [
'Hello World'
]
count => 10
}
}

output {
microsoft-sentinel-log-analytics-logstash-output-plugin {
client_id => "${SENTINEL_CLIENT_APP_ID}"
client_secret => "${SENTINEL_CLIENT_APP_SECRET}"
data_collection_endpoint => "${SENTINEL_DATA_COLLECTION_ENDPOINT}"
dcr_id => ""
stream_name => ""
tenant_id => "${SENTINEL_TENANT_ID}"
}
microsoft-sentinel-log-analytics-logstash-output-plugin {
client_id => "${SENTINEL_CLIENT_APP_ID}"
client_secret => "${SENTINEL_CLIENT_APP_SECRET}"
data_collection_endpoint => "${SENTINEL_DATA_COLLECTION_ENDPOINT}"
dcr_immutable_id => ""
dcr_stream_name => ""
tenant_id => "${SENTINEL_TENANT_ID}"
}
}
```

Steps to reproduce the behavior:
1. Build each of the Dockerfiles
2. Run each of the Dockerfiles
3. Observe the difference in the Logstash logs

**Expected behavior**

The expected behaviour is a log that contains the DCR stream name and the request ID.

```
[2026-06-02T14:35:18,896][INFO ][logstash.outputs.microsoftsentineloutput][sentinel][7bcdf1ba19752d96e8a18fa0a2834db6b8a6d1dee5c4a156e0be2af069c6e02d] Successfully posted 100 logs into log analytics DCR stream [Custom-TableName_CL] x-ms-request-id [xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx].
```

However, the logs for v2.2.1 look like the following.

```
[2026-06-02T15:07:14,588][INFO ][org.logstashplugins.LogAnalyticsEventsHandler.Workers.SenderWorker] Batch uploaded successfully. Batch size: 2962. Thread id: 39
```

**Screenshots**
N/A

**Desktop (please complete the following information):**
- OS: N/A
- Browser N/A
- Version N/A

**Smartphone (please complete the following information):**
- Device: N/A
- OS: N/A
- Browser N/A
- Version N/A

**Additional context**

This has been observed using a dockerised Logstash setup using the base image `docker.elastic.co/logstash/logstash:8.19.16`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.