Failed to handle valid XML request body larger than ... | owasp-modsecurity/ModSecurity-nginx

Nobody has claimed this yet.

How to reproduce:

Generate some valid XML larger than XML_MAX_LOOKUP_LIMIT (https://github.com/GNOME/libxml2/blob/master/include/libxml/parserInternals.h#L73, 10000000 bytes by default) e.g. with this script: https://gist.github.com/defanator/e06a66a3275fd75b39c37d6a00f7b7a4

vagrant@vagrant:~$ ./generate-xml.sh 26700 > test.xml
vagrant@vagrant:~$ stat test.xml 
  File: test.xml
  Size: 10333790  	Blocks: 20184      IO Block: 4096   regular file
Device: 803h/2051d	Inode: 3418121     Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/ vagrant)   Gid: ( 1000/ vagrant)
Access: 2021-05-05 10:30:46.459915604 +0000
Modify: 2021-05-05 10:30:44.127915604 +0000
Change: 2021-05-05 10:30:44.127915604 +0000
 Birth: -

Push it to nginx running with ModSecurity-nginx connector:

vagrant@vagrant:~$ curl -v -d @test.xml -H "Content-Type: text/xml" http://localhost/modsec-full/upload/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> POST /modsec-full/upload/ HTTP/1.1
> Host: localhost
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Type: text/xml
> Content-Length: 10013375
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 400 Bad Request
< Server: nginx/1.19.10
< Date: Wed, 05 May 2021 10:30:56 GMT
< Content-Type: text/html
< Content-Length: 158
< Connection: close
< 
<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx/1.19.10</center>
</body>
</html>
* Closing connection 0

The logs will contain something like this:

2021/05/05 10:30:56 [error] 20269#20269: *4 [client 127.0.0.1] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/nginx/modsec/modsecurity.conf"] [line "10"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data "XML parsing error: XML: Failed parsing document."] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri "/modsec-full/upload/"] [unique_id "1620210656"] [ref "v156,1"], client: 127.0.0.1, server: , request: "POST /modsec-full/upload/ HTTP/1.1", host: "localhost"

But the actual error message from libxml2 will be:

body.xml:1: parser error : internal error: Huge input lookup
      <time>2009-10-17T18:37:34Z</time>      </trkpt>    </trkseg>  </trk></gpx>
                                                                               ^

(see https://github.com/SpiderLabs/ModSecurity/issues/2565 for details on how to get the actual message)

I see that the library has some functions to process XML payload chunk by chunk, so I suspect there's some limitation at the connector's side (thus I'm filling this issue in the connector project).

Start by reproducing the request with the generated XML larger than XML_MAX_LOOKUP_LIMIT through the ModSecurity-nginx connector, then inspect how the connector passes request-body chunks to libxml2. Done means valid XML of this size is handled without the XML parsing error and the request receives the expected result.

Written by the indexing model from the issue text.

Failed to handle valid XML request body larger than XML_MAX_LOOKUP_LIMIT

Description

Contributor guide

First steps

Research direction

Assessment