owasp-modsecurity / owasp-modsecurity/ModSecurity

Performance issues with ModSecurity2/Alpine/PCRE2

Open
#3,409 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

2.x
Dominant language
C++
Stars
9.8k
Forks
1.8k
Avg merge
2h 46m
Merged PRs (30d)
1

Description

Describe the bug

When using PCRE2 instead of PCRE1 there is a significant performance loss with ModSecurityV2 on Alpine Linux compared to Debian.
A ModSec2 installation with Apache on Alpine Linux handles requests much slower and is using up considerable amounts of CPU ressources.
This behavior can be reproduced using the official OWASP CRS container image: https://github.com/coreruleset/modsecurity-crs-docker
The test described below results in about 5% to 10% higher CPU usage on the alpine image compared to the debian image.
In this screenshot from btop you can see the difference in the CPU usage for the same workfload(left debian, right alpine):

Image

This difference seems to be dependent on the exact usecase. In one of our production setups we had about 10 times higher CPU loads under certain conditions. This even lets the OpenShift healthchecks fail for these containers due to timeouts.
I focus on the simple request in this issue, because our productive setup is much more complicated to reproduce.

@airween is already aware of this issue, but was unable to reproduce it until now.

Logs and dumps

No problems discoverable in the logs

To Reproduce

The issue can be reproduced with the official OWASP-ModSecurity Docker image. I used the following Docker run commands to start the container with either the most current debian or Alpine image.

Debian: owasp/modsecurity-crs:apache
Alpine: owasp/modsecurity-crs:apache-alpine

Docker run command:

docker run \
   -ti \
   -p 9090:8080 \
   --rm \
   -v /tmp/host-fs-auditlog.log:/var/log/modsec_audit.log \
   -v /tmp/host-fs-errorlog.log:/var/log/modsec_error.log \
   -e MODSEC_AUDIT_ENGINE=on \
   -e LOGLEVEL=warn \
   -e ERRORLOG=/var/log/modsec_error.log \
   -e BLOCKING_PARANOIA=2 \
   -e DETECTION_PARANOIA=2 \
   -e ENFORCE_BODYPROC_URLENCODED=1 \
   -e ANOMALY_INBOUND=10 \
   -e ANOMALY_OUTBOUND=5 \
   -e ALLOWED_METHODS="GET HEAD POST OPTIONS PUT PATCH DELETE" \
   -e ALLOWED_REQUEST_CONTENT_TYPE="application/x-www-form-urlencoded|multipart/form-data|multipart/related|text/xml|application/xml|application/soap+xml|application/json|application/cloudevents+json|application/cloudevents-batch+json|text/plain" \
   -e ALLOWED_REQUEST_CONTENT_TYPE_CHARSET="utf-8|iso-8859-1" \
   -e ALLOWED_HTTP_VERSIONS="HTTP/1.1 HTTP/2 HTTP/2.0" \
   -e RESTRICTED_EXTENSIONS=".cmd/ .com/ .config/ .dll/" \
   -e RESTRICTED_HEADERS="/proxy/ /if/" \
   -e STATIC_EXTENSIONS="/.jpg/ /.jpeg/ /.png/ /.gif/" \
   -e MAX_NUM_ARGS=128 \
   -e ARG_NAME_LENGTH=50 \
   -e ARG_LENGTH=200 \
   -e TOTAL_ARG_LENGTH=6400 \
   -e MAX_FILE_SIZE=100000 \
   -e COMBINED_FILE_SIZES=1000000 \
   -e TIMEOUT=60 \
   -e SERVER_ADMIN=root@localhost \
   -e SERVER_NAME=localhost \
   -e PORT=8080 \
   -e MODSEC_RULE_ENGINE=on \
   -e MODSEC_REQ_BODY_ACCESS=on \
   -e MODSEC_REQ_BODY_LIMIT=13107200 \
   -e MODSEC_REQ_BODY_NOFILES_LIMIT=131072 \
   -e MODSEC_RESP_BODY_ACCESS=off \
   -e MODSEC_RESP_BODY_LIMIT=524288 \
   -e MODSEC_PCRE_MATCH_LIMIT=1000 \
   -e MODSEC_PCRE_MATCH_LIMIT_RECURSION=1000 \
   -e VALIDATE_UTF8_ENCODING=1 \
   -e CRS_ENABLE_TEST_MARKER=1 \
   -e PROXY_SSL_VERIFY=off \
   -e PROXY_SSL=on \
   -e BACKEND="https://httpbin.org/" \
   owasp/modsecurity-crs:apache-alpine

I generated the load with a small shell script, that performs about 20 curl requests with some cookies per second:

#!/bin/bash

while [ true ] 
do

sleep 0.05

curl -s -k "http://<ip or url of running waf>:9090/status/200" -H "Cookie: NG_TRANSLATE_LANG_KEY=en; _n2os-webconsole_session-33564f983wewretgdfghdjfg5e14bf3f70901b9430f3b=b0d87ec87b9dfgdfgwerertw08cb0c89; XSRF-TOKEN-2dsfds86c-1be4-473b-b10f-6f7sdfwe589b2=_of55PO8gv3YZLNijkqp9bAMi_yEEVbIBXbHDE1x274Mlp_J72-FnWiocfPZVgR2bTwpAsd6XmOXHmMAldogoQ; atuserid=%7B%22name%22%3A%22atuserid%22%2C%22val%22%3A%werw-4a51-4ac4-a80e-c142ewrwerw%22options%22%3A%7B%22end%22%3A%werwe-07-15T05%3A56%3A01.951Z%22%2C%22path%22%3A%22%2F%22%7D%7D; atidvisitor=%7B%22name%22%3A%22atidvisitor%22%2C%22val%22%3A%7B%22vrn%22%3A%22-werwerwe234324-%22%7D%2C%22options%22%3A%7B%22path%22%3A%22%2F%22%2C%22session%22%23423423%2C%22end%22%23423423%7D%7D; 3db4ccd6a2c6234234234423_n2os-webconsole_session-ewrewr2342356235werwerwer23451234=856020593f6a7d6a62b9ba4ef6afea7c; XSRF-TOKEN-f66e8519-be08-4679-8995-9623423423wec27171=DdCIiYRrsDD1A53RHeVSRoFCc5FMHfeuaAleBP7_40Zpcy0wDDFzTd__UsMQ3MT234234ZeSN-5QAPLCumOUiyw" -w "%{time_total}\n" -o /dev/null >> ./$1

done

Expected behavior

The expected behavior would be a similar performance of the debain and alpine images.

Server (please complete the following information):

  • ModSecurity version 2.9.10
  • WebServer: apache-2.4.63
  • Debian and Alpine container images
  • Rocky Linux 9.6 docker-ce host system

Rule Set (please complete the following information):

  • CRS 4.15.0

Additional context

If further information is needed, please feel free to ask

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported comparison with the OWASP ModSecurity CRS Debian and Alpine images, using the provided Docker command and shell load script. Compare CPU usage and request timings, then trace the ModSecurity 2.9.10 Apache path involving PCRE2; done means identifying the cause and demonstrating comparable performance or documenting a confirmed limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
apache, cpp, docker, linux
Domain
devops, performance, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.