splunk / splunk/splunk-library-javalogging

Order of log entries not maintained

Open
#261 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
140
Forks
173
PR merge metrics
No merged PRs in 30d

Description

When we log multiple events during the same millisecond the order that they're logged isn't maintained when viewing those events in Splunk. Note that we're using a batch_size_count (which should be batchSizeCount, but that's a different issue) of 1 since we want to increase the chance that all events get persisted to Splunk.

logback.xml exhibiting the issue:

<configuration>
<appender name="splunk" class="com.splunk.logging.HttpEventCollectorLogbackAppender">
<url>${SPLUNK_URL}</url>
<token>${SPLUNK_TOKEN}</token>
<index>${SPLUNK_INDEX}</index>
<host>${SPLUNK_HOST}</host>
<connectTimeout>2000</connectTimeout>
<writeTimeout>1000</writeTimeout>
<batch_size_count>1</batch_size_count>
<includeMDC>false</includeMDC>
<includeThreadName>false</includeThreadName>

<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
<jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
<prettyPrint>false</prettyPrint>
</jsonFormatter>

<includeTimestamp>false</includeTimestamp>
<includeLevel>false</includeLevel>
<includeLoggerName>false</includeLoggerName>
<includeMDC>false</includeMDC>
<includeThreadName>false</includeThreadName>
<includeContextName>false</includeContextName>
</layout>
</appender>

<root level="info">
<appender-ref ref="splunk" />
</root>
</configuration>
`

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the logback.xml configuration shown, especially the Splunk HttpEventCollectorLogbackAppender and batch_size_count setting. Reproduce multiple events logged within the same millisecond and inspect their order in Splunk. Done means the persisted events retain their logging order without removing the batch_size_count=1 configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.