elastic / elastic/ecs-logging-java

MDC values for event.type and event.category are not properly serialized as JSON arrays

Open
#301 3 comments 0 reactions 0 assignees View on GitHub
agent-java community enhancement
Dominant language
Java
Stars
148
Forks
82
Avg merge
2d 17h
Merged PRs (30d)
3

Description

## Description
When using logback-ecs-encoder with SLF4J's MDC to set array-type fields (like event.type and event.category), the values are serialized as string literals instead of proper JSON arrays. While MDC only supports String values by design, the ECS encoder could detect and properly format string values that represent arrays for fields that are defined as arrays in the ECS specification.

## Current Behavior
When setting an array value in MDC (which only accepts strings):
```java
MDC.put("event.type", Arrays.asList("connection", "allowed").toString());
```

The current output in logs:

```json
{
"@timestamp": "2025-01-08T13:00:53.318Z",
"event.type": "[connection, allowed]",
// other fields...
}
```

## Expected Behaviour
The log output should contain a proper JSON array according to ECS specification:
```json
{
"@timestamp": "2025-01-08T13:00:53.318Z",
"event.type": ["connection", "allowed"],
// other fields...
}
```

## Technical Details
The issue is in `EcsJsonSerializer.serializeMDC()` where all MDC values are treated as string literals:
```java
builder.append("\":\"");
JsonUtils.quoteAsString(toNullSafeString(String.valueOf(entry.getValue())), builder);
builder.append("\",");
```

While we understand that MDC only supports string values, the ECS encoder could detect and properly format these string values for fields that are defined as arrays in the ECS specification.

## Impact
This limitation affects any field that should be an array according to ECS specification, particularly:

- event.type
- event.category
- tags
- labels

This makes it difficult to use the library with standard Java collections for fields that should be arrays according to the ECS specification.

## Suggested Solution
The serializer could:

- Check if the field name matches known array fields from ECS specification
- Check if the string value represents a list (e.g., starts with '[' and ends with ']')
- Parse and format such values as proper JSON arrays

Example implementation approach:

```java
private static final Set ARRAY_FIELDS = Set.of(
"event.type",
"event.category",
"tags",
"labels"
);

public static void serializeMDC(StringBuilder builder, Map properties) {
if (properties != null && !properties.isEmpty()) {
for (Map.Entry entry : properties.entrySet()) {
builder.append('\"');
String key = entry.getKey();
JsonUtils.quoteAsString(key, builder);

String value = toNullSafeString(String.valueOf(entry.getValue()));
if (value.startsWith("[") && value.endsWith("]")) {
List items = Arrays.stream(
value.substring(1, value.length() - 1)
.split(","))
.map(String::trim)
.collect(Collectors.toList());

builder.append("\":");
builder.append(formatAsJsonArray(items));
builder.append(",");
} else {
builder.append("\":\"");
JsonUtils.quoteAsString(toNullSafeString(String.valueOf(entry.getValue())), builder);
builder.append("\",");
}
}
}
}
```

## Environment

- logback-ecs-encoder version: 1.6.0
- slf4j-api version: 2.0.9
- Java version: 11
- Logback version: 1.4.12

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.