splunk / splunk/splunk-library-javalogging

Some log entries missing

Open
#282 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
140
Forks
173
PR merge metrics
No merged PRs in 30d

Description

Hi,

I am using SplunkHttp appender with log4j2 for my Java console application. In case I am using batch mode (any batch_size_count bigger than 1) some of my entries logged before the shutdown are missing on splunk. For instance I expect to see [INFO ] 2023-12-11 11:35:27.626 [main] ExitCodePrinter - Application finished with exit code=1 but no luck.

The same application has console and file appenders and the log entry is not missing in console and log file.

This is my setup.
Dependency: com.splunk.logging:splunk-library-javalogging:1.11.8.
Log4j2 file:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="info" name="example" packages="com.splunk.logging">

    <Appenders>

        <SplunkHttp name="http"
                    url="***"
                    token="***"
                    index="***"
                    host="***"
                    sourcetype="log4j"
                    source="***"
                    messageformat="json">
            <errorCallback>com.splunk.logging.util.StandardErrorCallback</errorCallback>
            <JSONLayout compact="true" eventEol="true" stacktraceAsString="true" />
        </SplunkHttp>

        <File name="logfile" fileName="****" createOnDemand="true">
            <PatternLayout pattern="******"/>
        </File>

        <Console name="console" target="SYSTEM_OUT" follow="true">
            <PatternLayout pattern="***" />
        </Console>

    </Appenders>
    <Loggers>
        <Root level="INFO">
            <AppenderRef ref="console"/>
            <AppenderRef ref="logfile"/>
            <AppenderRef ref="http"/>
        </Root>
    </Loggers>

</Configuration>

This issue forces me to use batch_size_count=1 so that I don't loose some important logs. Would be nice being able to use batching due to performance reasons.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the missing-entry behavior with the SplunkHttp appender, Log4j2 configuration shown here, and batch_size_count greater than 1, then compare it with batch_size_count=1. Done means entries logged before shutdown reliably appear in Splunk while batching remains enabled; the issue does not name a source file or test to inspect.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.