owasp-modsecurity / owasp-modsecurity/ModSecurity

"Database is not open. Use: SecGeoLookupDb directive." after reload

Open
#2,041 14 comments 1 reaction 1 assignee View on GitHub

@victorhora is already working on this.

Since Mar 8, 2019.

3.x enhancement workaround available
Dominant language
C++
Stars
9.8k
Forks
1.8k
Avg merge
2h 46m
Merged PRs (30d)
1

Description

Describe the bug

I'm using mmdb to block countries from my website and it works fine until I reload my nginx server and I start getting the message "Database is not open. Use: SecGeoLookupDb directive." in the debug log and no country would be blocked anymore.

Logs and dumps

Before reload:

[...] [/] [4] (Rule: 20) Executing operator "GeoLookup against REMOTE_ADDR.
[...] [/] [9] Target value: "111.111.111.111" (Variable: REMOTE_ADDR)
[...] [/] [9] Matched vars updated.
[...] [/] [9] Saving msg: Block China IP address
[...] [/] [4] Rule returned 1.
[...] [/] [4] Executing chained rule.
[...] [/] [4] (Rule: 0) Executing operator "Within" with param "CN CL" Was: "" against GEO:COUNTRY_CODE.
[...] [/] [9] Target value: "CN" (Variable: GEO:COUNTRY_CODE)
[...] [/] [9] Matched vars updated.
[...] [/] [4] Rule returned 1.

After reload:

[...] [...] [4] (Rule: 20) Executing operator "GeoLookup against REMOTE_ADDR.
[...] [...] [9] Target value: "111.111.111.111" (Variable: REMOTE_ADDR)
[...] [...] [4] Database is not open. Use: SecGeoLookupDb directive.
[...] [...] [4] Rule returned 0.

ModSecurity config log:

ModSecurity - v3.0.3-56-g6d5198b1 for Linux
Mandatory dependencies

  • libInjection ....v3.0.3-56-g6d5198b1
  • SecLang tests ....6d5198b1

Optional dependencies

  • GeoIP/MaxMind ....found
    • (MaxMind) v1.3.2
      -lmaxminddb, -DWITH_MAXMIND -I/usr/local/include
    • (GeoIP) v1.6.12
      -lGeoIP, -I/usr/include/
  • LibCURL ....found v7.58.0
    -lcurl, -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL
  • YAJL ....found v2.1.0
    -lyajl, -DWITH_YAJL -I/usr/include/yajl
  • LMDB ....not found
  • LibXML2 ....found v2.9.4
    -lxml2, -I/usr/include/libxml2 -DWITH_LIBXML2
  • SSDEEP ....not found
  • LUA ....found v501
    -llua5.1 -L/usr/lib/x86_64-linux-gnu/, -DWITH_LUA -DWITH_LUA_5_1 -I/usr/include/lua5.1

Other Options

  • Test Utilities ....enabled
  • SecDebugLog ....enabled
  • afl fuzzer ....disabled
  • library examples ....enabled
  • Building parser ....disabled
  • Treating pm operations as critical section ....disabled

To Reproduce

Use the following configuration with your own country code and restart nginx.

SecGeoLookupDb /mnt/nginx/defaults/geoip/geoip.mmdb

SecAction
"id:19,
phase:1,
nolog,
pass,
t:none,
setvar:'tx.high_risk_country_codes=CN CL'"
SecRule REMOTE_ADDR "@geoLookup" "chain,id:20,drop,msg:'Block China IP address',phase:2"
SecRule GEO:COUNTRY_CODE "@within %{tx.high_risk_country_codes}"

Test with your own country code and you should get a 403 forbidden error code, then remove your country from the collection, reload and try again, you should receive 200. Add your country code again to the collection and you should get a 200 status and the previous reported message in the logs as the geoip database was unloaded.
Or you can just create a disruptive rule with geoip and after testing it works, you can check if it works after a reload.

Expected behavior

Allow me to reload the geoip database without restarting the server.

Server:

  • ModSecurity version (and connector): ModSecurity master (6d5198b), Connector master (d7101e1)
  • WebServer: 1.15.19
  • OS: 4.15.0-1021-aws #21-Ubuntu x86_64
  • libmaxminddb: 1.3.2 (from source)
  • Nginx flags: ./configure
    --with-http_ssl_module
    --add-module=/tmp/ModSecurity-nginx/
    --with-http_ssl_module
    --with-http_sub_module
    --with-http_realip_module
    --with-http_sub_module
    --with-http_gunzip_module
    --with-http_gzip_static_module

More info

It seems like Utils::GeoLookup::getInstance().cleanUp(); is being called and Utils::GeoLookup::setDataBase is not being called again, I'm trying to find out the problem over there.

Small workaround

You can "reload" nginx by renewing the process:

create new master and workers

pid=$(cat /usr/local/nginx/logs/nginx.pid)
sudo kill -USR2 $pid

gracefully kill the old workers

sudo kill -WINCH $pid

gracefully kill the old master

sudo kill -QUIT $pid

That would reload the whole engine and nginx without any downtime but using many resources.
Benchmarks for 1000 reloads and winchs in a 8 core 16gb memory server with just 100 rules:

#https://gist.github.com/jptosso/aa72ff932d3fad39614923bccb2d175a
ubuntu@server:~$ sudo bash /etc/test.sh
winch: 17 seconds
reload: 35 seconds

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.