apache / apache/logging-log4j2
In NoSql appender, be able to use both default fields and custom fields
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.7k
- Avg merge
- 21h 30m
- Merged PRs (30d)
- 27
Description
Right now, the fields of the documents that the NoSql appender creates are fixed, unless you use a MapMessage and a MessageLayout. In this case you get the fields from the MapMessage but none of the default fields.
If you do this (Kotlin code in this example, but Java would have the same problem):
``` Kotlin
logger.info(MapMessage(mapOf("aa" to "11", "bb" to "22")))
```
You can either get this:
``` JSON
{
"_id": {
"$oid": "64f1fc8ec553ec3593274f30"
},
"level": "INFO",
"loggerName": "InjectedLogger.-1169081147",
"message": "aa=\"11\" bb=\"22\"",
"source": {
"className": "InjectedLogger",
"methodName": "execute",
"fileName": "Main.kt",
"lineNumber": 32
},
"marker": null,
"threadId": {
"$numberLong": "1"
},
"threadName": "main",
"threadPriority": 5,
"millis": {
"$numberLong": "1693580430801"
},
"date": {
"$date": "2023-09-01T15:00:30.801Z"
},
"thrown": null,
"contextMap": {},
"contextStack": [],
"additionalFields": {}
}
```
Or this:
``` JSON
{
"_id": {
"$oid": "64f1dbbddb2eb17fd4809d51"
},
"aa": "11",
"bb": "22",
"additionalFields": {}
}
```
I want this (order of the fields is unimportant):
``` JSON
{
"_id": {
"$oid": "64f1fc8ec553ec3593274f30"
},
"level": "INFO",
"loggerName": "InjectedLogger.-1169081147",
"message": "aa=\"11\" bb=\"22\"",
"source": {
"className": "InjectedLogger",
"methodName": "execute",
"fileName": "Main.kt",
"lineNumber": 32
},
"marker": null,
"threadId": {
"$numberLong": "1"
},
"threadName": "main",
"threadPriority": 5,
"millis": {
"$numberLong": "1693580430801"
},
"date": {
"$date": "2023-09-01T15:00:30.801Z"
},
"thrown": null,
"contextMap": {},
"contextStack": [],
"additionalFields": {},
"aa": "11",
"bb": "22",
}
```
I have been doing some analysis and I believe that this functionality would be very difficult to achieve with the provided extension points, as the code that creates the document layout can't be swapped from configuration.
Then it would be easy to create a custom Message or a Layout that can control the message field and set it to something more interesting than repeating the custom properties (not part of this request). StructuredDataMessage, for instance, already has a message property that can be used via a custom layout.
I propose creating a configuration option that would enable this behavior.
I have been looking at the code and, apart from creating the configuration option, only a minor change would be required [here](https://github.com/apache/logging-log4j2/blob/d61f12bbd6318c3d1e1f086a1e5fcef353d31fb4/log4j-core/src/main/java/org/apache/logging/log4j/core/appender/nosql/NoSqlDatabaseManager.java#L260):
``` Java
if (serializable instanceof MapMessage) {
// Add these three lines
if (newOption){
setFields(event, entity);
}
setFields((MapMessage) serializable, entity);
} else {
setFields(event, entity);
}
```
Another option (maybe more flexible) would be to move all the code that sets the document properties from the event (`writeInternal` and `setFields` methods from `NoSqlDatabaseManager`) to a new class that can be swapped via configuration. Something in the lines of a `NoSqlObjectLayout` that would be conceptually similar to Layouts but that outputs (or sets fields to) a NoSql document instead of a `String`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in log4j-core/src/main/java/org/apache/logging/log4j/core/appender/nosql/NoSqlDatabaseManager.java, especially writeInternal and the setFields methods. Review how MapMessage handling and configuration options are implemented, then define the option's behavior and verify that default event fields and custom fields are both retained in the NoSQL document.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100