OpenConext / OpenConext/OpenConext-deploy

Json logging (for example for oidng) is broken

Open
#669 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jinja
Stars
16
Forks
20
Avg merge
4d 23h
Merged PRs (30d)
2

Description

The json logging does not work correctly. The logstash appender that is for example used in in roles/oidcng/templates/logback.xml.j2 looks like this

 <appender name="JSON_SYSLOG" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
   <destination>host.docker.internal:514</destination>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder">
        <customFields>{"app":"oidcng"}</customFields>
        <includeCallerData>true</includeCallerData>
        <fieldNames>
            <thread>[ignore]</thread>
            <version>[ignore]</version>
            <levelValue>[ignore]</levelValue>
       </fieldNames>
     </encoder>
  </appender>

This correctly format the json, but sends a Logstash-compatible message to a syslog port. In particular, it send pure json, whereas syslog instead expects an RFC3164 or RFC5452-compatible message. Such a syslog message contains a header followed by a message:

<PRIO> VERSION TIMESTAMP HOSTNAME APPNAME PROCID MSGID STRUC MESSAGE

for example:

<132> 1 20260507T12:00:00.000000+0200 docker1.dev.openconext.org oidcngjson - - - {"app":"oidng","more":"jsonfields"}

Using the current appender, rsyslog randoml interprets some of the json fields as its metadata fields cause for example the "servicename" to be {"@timestamp":.

This makes it impossible to match log messages and redirect them to the correct files.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with roles/oidcng/templates/logback.xml.j2 and compare the configured Logstash TCP appender with the syslog format described in the issue. Verify that the resulting messages are accepted as syslog by rsyslog, preserve the JSON payload, and can be routed to the correct files.

Written by the indexing model from the issue text.

Assessment

Tech stack
ansible, java
Domain
devops, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.