splunk / splunk/splunk-library-javalogging
Logback Appender and message size
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 140
- Forks
- 173
- PR merge metrics
- No merged PRs in 30d
Description
Hi all,
we are using HttpEventCollectorLogbackAppender in raw mode and it looks like sent messages are somehow cut at a bit less than 10240 bytes.
Checking source code I see there is a limit in batchSize (coincidential?) at 10240 and I cannot override it as we are in raw mode[1]: is there any tip around this issue? Sometime we need to send large-ish json payload in the message and we are losing informations.
This is our logback.xml configuration:
<appender name="SPLUNK_APPENDER"
class="com.splunk.logging.HttpEventCollectorLogbackAppender">
<url>${SPLUNK_URL}</url>
<token>${SPLUNK_TOKEN}</token>
<index>${SPLUNK_INDEX}</index>
<!-- disableCertificateValidation should be true as we do not use valid certificates -->
<disableCertificateValidation>true</disableCertificateValidation>
<source>${SOURCE}</source>
<sourcetype>logback</sourcetype>
<messageFormat>text</messageFormat>
<type>raw</type>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX, UTC} %.-3level SPAN_ID=%X{span_id} TRACE_ID=%X{trace_id} callId=%X{CALL_ID} uID=%X{USER_ID} ST=%X{SESSION_TOKEN} CH=%X{CHANNEL} %logger{35} - %msg%n%xException</pattern>
</layout>
</appender>
Thanks in advance for any help
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/main/java/com/splunk/logging/HttpEventCollectorLogbackAppender.java at the batchSize limit referenced in the issue, then trace how raw mode handles the configured Logback message. Reproduce the behavior with the supplied logback.xml and a JSON payload larger than 10240 bytes. Done means the supported behavior for large raw messages is clear and the payload is no longer unexpectedly truncated, or an explicit workaround is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100