logzio / logzio/logzio-java-sender

Spring Boot app crashes on start-up when logz.io appender is active

Open
#43 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
15
Forks
4
Avg merge
12m
Merged PRs (30d)
3

Description

One of our Spring Boot applications crashes when the logz.io appender is activated through the `logback-spring.xml`. Unfortunately, we weren't able to debug our issue. It only occurs when we deploy the app to a pod on GKE and not on our local Docker container.

Here's the stacktrace that was logged by our pod:
```
12:23:49.856 [main] ERROR o.s.boot.SpringApplication - Application run failed
java.lang.IllegalStateException: Logback configuration error detected:
ERROR in ch.qos.logback.core.joran.spi.Interpreter@70:20 - RuntimeException in Action for tag [appender] java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@51650883[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@1a15b789[Wrapped task = io.logz.sender.LogzioSender$$Lambda$214/1201004240@57f791c6]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@6c4f9535[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 9]
at org.springframework.boot.logging.logback.LogbackLoggingSystem.loadConfiguration(LogbackLoggingSystem.java:169)
at org.springframework.boot.logging.AbstractLoggingSystem.initializeWithConventions(AbstractLoggingSystem.java:82)
at org.springframework.boot.logging.AbstractLoggingSystem.initialize(AbstractLoggingSystem.java:60)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.initialize(LogbackLoggingSystem.java:117)
at org.springframework.boot.context.logging.LoggingApplicationListener.initializeSystem(LoggingApplicationListener.java:264)
at org.springframework.boot.context.logging.LoggingApplicationListener.initialize(LoggingApplicationListener.java:237)
at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEnvironmentPreparedEvent(LoggingApplicationListener.java:200)
at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEvent(LoggingApplicationListener.java:173)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:74)
at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:54)
at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:338)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:297)
at de.meindach.B2bApplicationKt.main(B2bApplication.kt:22)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
```
Appender configuration:
```

****
https://listener.logz.io:8071
java
true






{
"env": "****",
"appname": "****"
}




loglevel


logger




30
2048
20
^sun\.reflect\..*\.invoke
^net\.sf\.cglib\.proxy\.MethodProxy\.invoke
true





```
Relevant Maven dependencies:
```

io.logz.logback
logzio-logback-appender
1.0.22


net.logstash.logback
logstash-logback-encoder
5.3

```
After removing all `` occurrences from `logback-spring.xml` the app successfully starts.

How can I find out what goes wrong in the `io.logz.sender.LogzioSender` class?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with io.logz.sender.LogzioSender and the LogzioLogbackAppender initialization path implicated by the stacktrace, then compare the logback-spring.xml configuration and Maven versions shown here between the GKE pod and local Docker container. Reproduce the startup failure with the LOGZIO appender enabled and verify that initialization no longer rejects scheduled tasks and the application starts successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, google-cloud, java, spring-boot
Domain
backend, devops, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.