Log entries truncated

Describe the bug

Error log entries are truncated if the error log line is long enough. This cuts off information like the rule tags, sometimes even severity etc.

This comes up in the context of CRS where tags are used extensively. However, the CRS project cannot do much to mitigate this.

Logs and dumps
Here is a selection of rules where the tags are truncated. Note that sometimes the tags are truncated to different lengths for the same rule.

[user@host ~]$ cat /path/to/error.log | grep -v "event-correlation" | egrep "\[tag" | sed 's/.*\(\[id \S\+\]\).*\?\(\[tag.*\)/\1 \2/' | sed 's/ \[hostname.*//' | egrep -v "\]$" | head -n 10000 | sort -u
[id "932110"] [tag "OWASP_CRS/WEB_ATTACK/COMMAND_INJECTION"]
[id "942130"] [tag "attack-sqli"] [t
[id "942200"] [tag
[id "942200"] [tag "OWASP_AppSensor/CIE1"] [
[id "942200"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJEC
[id "942210"] [tag
[id "942210"] [tag "capec/10
[id "942260"] [tag "WASCTC/WASC-
[id "942260"] [tag "WASCTC/WASC-19"] [
[id "942340"] [tag "OW
[id "942340"] [tag "OWASP_TO
[id "942340"] [tag "PCI
[id "942370"] [tag "OWASP_AppSensor/CIE1
[id "942370"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECT
[id "942370"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"]
[id "942430"] [tag "applicat
[id "942430"] [tag "application-multi"] [ta
[id "942430"] [tag "capec/
[id "942430"] [tag "paranoi
[id "942430"] [tag "paranoia-level/
[id "942430"] [tag "paranoia-level/2"
[id "942430"] [tag "PCI/6.5
[id "942430"] [tag "PCI/6.5.
[id "942450"] [tag "OWASP_AppSensor/CIE

Here is an example of a complete log entry:

[2020-09-29 03:59:11.378270] [-:error] 111.111.111.111:111111 XXXXXXXXXXXXXXXXXXXXXXXXXXX [client 111.111.111.111] ModSecurity: Warning. Pattern match "(?i:[\\\\s'\\"`()]*?\\\\b([\\\\d\\\\w]+)\\\\b[\\\\s'\\"`()]*?(?:<(?:=(?:[\\\\s'\\"`()]*?(?!\\\\b\\\\1\\\\b)[\\\\d\\\\w]+|>[\\\\s'\\"`()]*?(?:\\\\b\\\\1\\\\b))|>?[\\\\s'\\"`()]*?(?!\\\\b\\\\1\\\\b)[\\\\d\\\\w]+)|(?:not\\\\s+(?:regexp|like)|is\\\\s+not|>=?|!=|\\\\^)[\\\\s'\\"`()]*?(?!\\\\b\\\\1\\\\b)[\\\\d\\\\w]+|(?:(?:s ..." at XML. [file "/opt/modsecurity-rules/3.3.0/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "590"] [id "942130"] [msg "SQL Injection Attack: SQL Tautology Detected"] [data "Matched Data: ZZZZZZZZZZZZZZZZZZZZZZZZZ found within XML: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy"] [severity "CRITICAL"] [ver "OWASP_CRS/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-sqli"] [t [hostname "aaaaaaaaaaaaaaaaaaaaaa"] [uri "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"] [unique_id "XXXXXXXXXXXXXXXXXXXXXXXXXXX"]

Expected behavior

The log entries should not be truncated.

Alternatively, configuration options could be added that allow the user to configure the number of characters that are allowed before the log entry is truncated.

Server (please complete the following information):

• ModSecurity version: ModSecurity v2.9.3
• WebServer: Apache httpd 2.4.46
• OS: RHEL

Rule Set (please complete the following information):

• Running any public or commercial rule set? CRS
• What is the version number? 3.3.0

Additional context

See also: https://github.com/coreruleset/coreruleset/issues/1888.