elastic / elastic/logstash

Document how to configure logging for libraries or part of code that leverage Java Util Logging inside Logstash

Open
#16,193 2 comments 1 reaction 1 assignee Claimed by @andsel View on GitHub
docs
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

Logstash being a Java application uses JDK and in some circumstances that code log some messages that could be helpful to troubleshoot and investigate some issues. For example, HTTP client provided by the JDK logs some informations when issues HTTP requests or receive responses. However, those JDK libraries leverages JUL (Java Util Logging) to log the messages, and if not configured the user can't see anything.
Document how to configure JUL from Logstash perspective to obtain such log messages.

----

### Pre-existing knowledge

## How to enable JUL (Java Util Logging) in Logstash

- edit a file with loggers definitions name it `/conf/jul.properties`:
```
handlers= java.util.logging.ConsoleHandler,java.util.logging.FileHandler
.level= ALL
java.util.logging.FileHandler.pattern = /logs/jdk_http%u.log
java.util.logging.FileHandler.limit = 50000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.level=ALL
java.util.logging.FileHandler.maxLocks = 100
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter

java.util.logging.ConsoleHandler.level = INFO # or put FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter

# defines the logger we are interested
sun.net.www.protocol.http.HttpURLConnection.level = ALL
```
- then in their `config/jvm.options` configure the setting to point to the created config file, adds the following line:
```
-Djava.util.logging.config.file=/conf/jul.properties
```

Couple of refs:
- https://jenkov.com/tutorials/java-logging/configuration.html
- https://stackoverflow.com/a/8249319

An example from my local env, without any proxy:
```
FINEST: Looking for HttpClient for URL https://andselteststorageaccount.blob.core.windows.net/logstash/$Default/0 and proxy value of DIRECT
Jan 18, 2024 7:08:18 PM sun.net.www.http.ClientVector get
FINEST: cached HttpClient was idle for 0
Jan 18, 2024 7:08:18 PM sun.net.www.protocol.https.HttpsClient New
FINEST: KeepAlive stream retrieved from the cache, sun.net.www.protocol.https.HttpsClient(https://andselteststorageaccount.blob.core.windows.net/logstash/$Default/0?comp=lease)
Jan 18, 2024 7:08:18 PM sun.net.www.protocol.http.HttpURLConnection plainConnect0
FINEST: Proxy used: DIRECT
Jan 18, 2024 7:08:18 PM sun.net.www.protocol.http.HttpURLConnection writeRequests
FINE: sun.net.www.MessageHeader@6d61d3217 pairs: {PUT /logstash/$Default/0 HTTP/1.1: null}{Accept: application/xml}{Accept-Charset: UTF-8}{Content-Type: }{x-ms-version: 2019-02-02}{User-Agent: Azure-Storage/8.6.6 (JavaJRE 17.0.9; MacOSX 14.2.1)}....
Jan 18, 2024 7:08:18 PM sun.net.www.protocol.http.HttpURLConnection getInputStream0
FINE: sun.net.www.MessageHeader@1578b06012 pairs: {null: HTTP/1.1 201 Created}{Content-Length: 0}{Content-MD5: Z1NtaLEO+xEr6n1te7768g==}{Last-Modified: Thu, 18 Jan 2024 18:08:18 GMT}{ETag: "0x8DC185072D8F5CD"}{Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0}....
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.