Overriding incoming request IP

Hi everyone,

First of all, thanks for this awesome piece of software!

I'm trying to use OpenResty to dynamically update the incoming request IP based on a query parameter. I have an external service which is calling a given url on my server, and pass as argument the real ip. I need to route this request to another route on the same server, while updating the incoming IP. As a simple example, please see the following configuration:

log_format logf '$msec'             # ts
	     '\t$uri'                   # client
	     '\t$remote_addr'           # ip
	     '\t$http_referer'          # page
	     '\t$query_string'          # adcontrol
	     '\t$http_user_agent'       # uagent
	     '\t$server_port'           # port
	     '\t$http_x_forwarded_for'; # proxy

server {
    real_ip_header X-Forwarded-For;
    set_real_ip_from 0.0.0.0/0;

    listen       80 default_server;
    server_name _;

    error_log hmx_error.log;
    access_log hmx_access.log logf;

    location / {
        access_log off;

        access_by_lua_block {
            ngx.log(ngx.ERR, "Changing IP")
            ngx.req.set_header('X-Forwarded-For', '4.3.2.1')
            ngx.exec("/t")
        }
    }

    location = /t {

    }
}

The service calls the / route, which update the X-Forwarded-For header with the new IP, and the request is forwarded to the /t route. This works exactly as expected, see that the IP is correctly updated:

> curl localhost
nginx acces.log: 1526930357.285	/t	4.3.2.1	-	-	curl/7.54.0	80	4.3.2.1

However, my server is behind a load balancer, and the incoming request already has the X-Forwarded-For header set. In this configuration, it seems like updating the header doesn't do anything:

> curl -H "X-Forwarded-For: 1.2.3.4" localhost
nginx access.log: 1526930549.882	/t	1.2.3.4	-	-	curl/7.54.0	80	4.3.2.1

As you can see, the proxy IP is the correct one, but the remote IP is the one from the original X-Forwarded-For header, and not the one from the updated header. It looks like the real_ip_header handler is not called again when the header is updated.

I think this is an issue since it's working correctly if the header is not set in the first place. If you need more information from me, please just ask!

Again, thanks a lot for this project!

Openresty version (also tested on Linux with the same issue):

nginx version: openresty/1.13.6.2
built by clang 9.1.0 (clang-902.0.39.1)
built with OpenSSL 1.1.0h  27 Mar 2018
TLS SNI support enabled
configure arguments: --prefix=/usr/local/Cellar/openresty/1.13.6.2/nginx --with-cc-opt='-O2 -I/usr/local/include -I/usr/local/opt/pcre/include -I/usr/local/opt/openresty-openssl/include' --add-module=../ngx_devel_kit-0.3.0 --add-module=../echo-nginx-module-0.61 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2rc3 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.31 --add-module=../ngx_lua-0.10.13 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../ngx_stream_lua-0.0.5 --with-ld-opt='-Wl,-rpath,/usr/local/Cellar/openresty/1.13.6.2/luajit/lib -L/usr/local/lib -L/usr/local/opt/pcre/lib -L/usr/local/opt/openresty-openssl/lib' --pid-path=/usr/local/var/run/openresty.pid --lock-path=/usr/local/var/run/openresty.lock --conf-path=/usr/local/etc/openresty/nginx.conf --http-log-path=/usr/local/var/log/nginx/access.log --error-log-path=/usr/local/var/log/nginx/error.log --with-pcre-jit --with-ipv6 --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_geoip_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-dtrace-probes --with-stream --with-stream_ssl_module --with-http_ssl_module

Start by reproducing the configuration with an existing X-Forwarded-For header and compare the logged $remote_addr with the rewritten header after ngx.exec("/t"). Trace how real_ip_header is processed relative to access_by_lua_block and internal routing. Done means the expected behavior is documented or corrected and covered by a regression test for both absent and pre-existing headers.