splunk / splunk/splunk-library-javalogging
Order of log entries not maintained
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 140
- Forks
- 173
- PR merge metrics
- No merged PRs in 30d
Description
When we log multiple events during the same millisecond the order that they're logged isn't maintained when viewing those events in Splunk. Note that we're using a batch_size_count (which should be batchSizeCount, but that's a different issue) of 1 since we want to increase the chance that all events get persisted to Splunk.
logback.xml exhibiting the issue:
<configuration>
<appender name="splunk" class="com.splunk.logging.HttpEventCollectorLogbackAppender">
<url>${SPLUNK_URL}</url>
<token>${SPLUNK_TOKEN}</token>
<index>${SPLUNK_INDEX}</index>
<host>${SPLUNK_HOST}</host>
<connectTimeout>2000</connectTimeout>
<writeTimeout>1000</writeTimeout>
<batch_size_count>1</batch_size_count>
<includeMDC>false</includeMDC>
<includeThreadName>false</includeThreadName>
<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>false</prettyPrint>
</jsonFormatter>
<includeTimestamp>false</includeTimestamp>
<includeLevel>false</includeLevel>
<includeLoggerName>false</includeLoggerName>
<includeMDC>false</includeMDC>
<includeThreadName>false</includeThreadName>
<includeContextName>false</includeContextName>
</layout>
</appender>
<root level="info">
<appender-ref ref="splunk" />
</root>
</configuration>
`
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the logback.xml configuration shown, especially the Splunk HttpEventCollectorLogbackAppender and batch_size_count setting. Reproduce multiple events logged within the same millisecond and inspect their order in Splunk. Done means the persisted events retain their logging order without removing the batch_size_count=1 configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100