openresty / openresty/lua-nginx-module

Response body missing when using ngx.location.capture() to fetch response from a location that uses proxy_pass, cache & aio

Open
#587 15 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
11.8k
Forks
2.1k
Avg merge
6h 1m
Merged PRs (30d)
6

Description

I have a setup where requests are captured by a location that has a content_by_lua_file directive. The lua code here contains some pre and post processing, but the important bit is that it uses ngx.location.capture to make a request to another nginx location. This location is setup with proxy_pass and cache.

I wanted to measure the impact of the dedicated io threads feature that was added to nginx core recently and so recompiled openresty (1.9.3.1) with --with-threads and --with-file-aio flags passed to the configure script and after the rebuild and nginx restart, added the aio threads; directive to the second location that handles proxying and caching.

The result was that the response body was missing when I hit the location handled by lua. The headers are present. Response looks fine when I hit the cached location directly instead of hitting the lua location that uses ngx.localtion.capture

Here is what the example setup looks like:

location /location1 {
    content_by_lua_file "location1.lua";
}


location /location2 {
    rewrite                 /location2/(.*) /$1 break;
    proxy_pass              http://localhost:8081;

    proxy_cache             my_cache;
    proxy_cache_key         $uri$is_args$args;

    proxy_cache_lock        on;
    proxy_cache_valid       200 7d;
    proxy_ignore_headers    "Expires" "Cache-Control" "Set-Cookie";

    # add a cache hit status header
    more_set_headers "X-Nginx-Cache: $upstream_cache_status";

    # newly added
    aio     threads;
}

The contents of location1.lua look something like this:

local url = "/location2" .. ngx.var.uri
sr_response = ngx.location.capture(url)

-- additional logic

ngx.status = sr_response.status
ngx.print(sr_response.body)

The goal is to take advantage of nginx's HTTP caching capabilities and to execute some additional logic in lua based on the response.

Removing the aio directive makes it work as expected again. Same results with aio on; instead of aio threads;

Tried it with the error_log level set to debug, didn't see anything except for a couple lines that said route matched location2 and that the connection was closed after that.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the two nginx locations, the shown location1.lua, proxy_pass, cache, and aio enabled, then compare it with aio removed. Start by tracing how ngx.location.capture returns the proxied response body when the captured location uses asynchronous I/O. Done means the captured response includes its body with aio on or the limitation is documented with a confirmed explanation.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua, nginx
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.