openresty / openresty/lua-nginx-module

always_forward_body=true not working

Open
#2,369 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
11.8k
Forks
2.1k
Avg merge
6h 1m
Merged PRs (30d)
6

Description

My nginx has the following configuration (it forwards and mirrors the parent request to the /pgsql, /mongo locations, and receives the response result of /pgsql)

        http {
            ......
            log_subrequest on;
            proxy_buffering off;
            proxy_request_buffering off;
            ......
        }

        location /test
        {
            error_log /data/logs/error.log debug;
            lua_need_request_body on;
            content_by_lua_block {
                local reqBody = ngx.req.get_body_data()
                local reqUri = ngx.var.uri or ""
                local resPg, resMongo = ngx.location.capture_multi( {
                                {"/pgsql".. reqUri, { method = ngx.HTTP_POST, body = reqBody , args = ngx.var.args, always_forward_body=true }},
                                {"/mongo".. reqUri , { method = ngx.HTTP_POST, body = reqBody , args = ngx.var.args, always_forward_body=true }}
                            })
                ngx.status = resPg.status
                for k, v in pairs(resPg.header) do
                    ngx.header[k] = v
                end
                ngx.print(resPg.body)
            }
        }

        location /pgsql
        {
            if ($request_method = 'OPTIONS') {
                add_header 'Access-Control-Allow-Origin' '*';
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
                add_header 'Access-Control-Allow-Headers' '*';
                add_header 'Access-Control-Expose-Headers' '*';
                add_header 'Content-Type' 'text/plain charset=UTF-8';
                add_header 'Content-Length' 0;
                return 204;
            }

            add_header 'Access-Control-Allow-Origin' '*';
            add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
            add_header 'Access-Control-Allow-Headers' '*';
            add_header 'Access-Control-Expose-Headers' '*';

            rewrite ^/pgsql/test$ $original_path break;
            proxy_pass http://xxx:xxx/test;
            proxy_http_version 1.1;
            proxy_set_header   Connection       "";
        }


        location /mongo
        {
            if ($request_method = 'OPTIONS') {
                add_header 'Access-Control-Allow-Origin' '*';
                add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
                add_header 'Access-Control-Allow-Headers' '*';
                add_header 'Access-Control-Expose-Headers' '*';
                add_header 'Content-Type' 'text/plain charset=UTF-8';
                add_header 'Content-Length' 0;
                return 204;
            }

            add_header 'Access-Control-Allow-Origin' '*';
            add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
            add_header 'Access-Control-Allow-Headers' '*';
            add_header 'Access-Control-Expose-Headers' '*';

            rewrite ^/mongo/test$ $original_path break;
            proxy_pass http://xxx:xxx/test;
            proxy_http_version 1.1;
            proxy_set_header   Connection       "";
        }

		......

I hope nginx can forward requests in a streaming manner (forward when receiving a request, forward when receiving a response), so I configured proxy_buffering off, proxy_request_buffering off, always_forward_body=true. But after capturing the packet, I found that

  • The subrequests of /pgsql and /mongo are forwarded only after receiving the body of the parent request;
  • The response result is also forwarded only after receiving the response of /pgsql;

I saw the documentation description:always_forward_body when set to true,...... The request body read by either ngx.req.read_body() or lua_need_request_body on will be directly forwarded to the subrequest without copying the whole request body data when creating the subrequest (no matter the request body data is buffered in memory buffers or temporary files).

Can someone take a look at and give me some guidance?

Thanks.

nginx version

/usr/local/openresty/nginx/sbin/nginx -V
nginx version: openresty/1.19.9.1
built by gcc 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC)
built with OpenSSL 1.1.1k  25 Mar 2021 (running with OpenSSL 1.1.1w  11 Sep 2023)
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-cc-opt='-O2 -DNGX_LUA_ABORT_AT_PANIC -I/usr/local/openresty/zlib/include -I/usr/local/openresty/pcre/include -I/usr/local/openresty/openssl111/include' --add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.20 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../ngx_stream_lua-0.0.10 --with-ld-opt='-Wl,-rpath,/usr/local/openresty/luajit/lib -L/usr/local/openresty/zlib/lib -L/usr/local/openresty/pcre/lib -L/usr/local/openresty/openssl111/lib -Wl,-rpath,/usr/local/openresty/zlib/lib:/usr/local/openresty/pcre/lib:/usr/local/openresty/openssl111/lib' --with-cc='ccache gcc -fdiagnostics-color=always' --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-compat --with-stream --with-http_ssl_module

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the supplied configuration using ngx.location.capture_multi, always_forward_body, and the proxy buffering settings, then compare packet capture timing for the parent body, subrequests, and responses. Read the ngx.location.capture_multi and request-body documentation first; done means determining whether streaming is supported here and documenting or correcting the observed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua, nginx
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.