splunk / splunk/splunk-library-javalogging
Requesting a Splunk-specific layout-class for log4j2
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 140
- Forks
- 173
- PR merge metrics
- No merged PRs in 30d
Description
By default -- without any layout specified -- the SplunkHttp-appender will simply log the textual message itself, along with a few other standard fields.
To get more details -- and even, optionally, insert additional custom fields -- one would use the JsonLayout. For example:
<SplunkHttp name="SplunkDetailed"
url="https://splunk-hec-app1:8088"
index="idx"
host="${hostname}"
sourcetype="log4j2"
source="QA:${nickname}:${alias}"
batch_size_count="17"
batch_interval="3"
disableCertificateValidation="true"
token="7d8xxx25b">
<JsonLayout
compact="true"
eventEol="true"
propertiesAsList="true"
includeStacktrace="true"
locationInfo="true"
objectMessageAsJsonObject="true"/>
<KeyValuePair key="applicationInstance" value="QA"/>
</JsonLayout>
</SplunkHttp>
This works Ok, but results in useless duplications... For example, here is one event logged using above configuration:
{
"severity": "WARN",
"logger": "our.deeply.buried.class.MessageHandler",
"time": "1632494387.518",
"thread": "pool-15-thread-2",
"message": {
"thread": "pool-15-thread-2",
"level": "WARN",
"loggerName": "our.deeply.buried.class.MessageHandler",
"message": "Our verbose message",
"endOfBatch": false,
"loggerFqcn": "org.slf4j.impl.Log4jLoggerAdapter",
"instant": {
"epochSecond": 1632494387,
"nanoOfSecond": 518362000
},
"applicationInstance": "QA",
"contextMap": [],
"threadId": 154,
"threadPriority": 5,
"source": {
"class": "our.deeply.buried.class.MessageHandler",
"method": "createImportPreconditionsMessage",
"file": "MessageHandler.java",
"line": 91
}
}
}
As you can see, there are problems:
message.levelis the same thing asseverity.message.threadis a copy ofthread.loggeris a copy ofmessage.loggerName.message.instantduplicatestime.endOfBatchis quite useless, but cannot be suppressed.message.contextMapis always included even when empty.- The useful additional information is all "hidden" in the
messagedictionary -- including the actualmessage.messagetext. It'd be nicer, if the fields were at the top level:threadPriorityinstead ofmessage.threadPriority.
One cannot blame the stock JsonLayout class for this redundancy, because it does not know, that the calling appender (SplunkHttp) is adding the information too. One's only way to make the events appear more sensible, currently, is to implement special manipulations on the Splunk-server side, based on the sourceType.
Hence a request for a custom SplunkLayout to format the messages as would make sense for Splunk in particular.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the SplunkHttp appender and the stock JsonLayout configuration described in the issue. Define the SplunkLayout output around the listed redundancies: avoid duplicated fields, omit unwanted empty or fixed fields, and expose useful message fields at the top level. Done means Splunk events have the requested non-redundant structure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100