openresty / openresty/lua-nginx-module

Cropped response when using body_filter_by_lua with sendfile and aio threads ("task ... already active")

Open
#2,118 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
11.8k
Forks
2.1k
Avg merge
6h 1m
Merged PRs (30d)
6

Description

I'm getting incomplete response from OpenResty when using body_filter_by_lua_block together with proxy_cache, sendfile and aio threads. In error log there's alert:

[alert] 1826080#1826080: task #1114 already active

Note 1: it happens only when the traffic goes through physical interface, can't reproduce on loopback
Note 2: remove any of body_filter_by_lua_block, sendfile on or aio threads and the problem no longer appears

Reproduced on OpenResty:

Ubuntu 22.04 and 20.04

nginx.conf:

error_log error.log;

worker_processes auto;

events {
    worker_connections 1024;
}

http {
    sendfile on;
    aio      threads;

    proxy_cache_path cache keys_zone=cache:10m;

    init_by_lua_block {
        static_data = ("."):rep(4000000)
    }
    server {
        listen 127.0.0.1:8001;
        access_log upstream.log;
        location = / {
            content_by_lua_block {
                ngx.header["Content-Length"] = static_data:len()
                ngx.print(static_data)
            }
        }
    }
    upstream upstream {
        server 127.0.0.1:8001;
    }

    server {
        listen 0.0.0.0:8080;

        access_log access.log;

        location / {
            body_filter_by_lua_block {
            }

            proxy_cache cache;
            proxy_pass http://upstream;
        }
    }
}

Run with:

openresty -p ./ -e error.log -c nginx.conf -g 'pid nginx.pid;'

Test with:

curl -sv host:8080 | wc -m

Expected to receive exactly 4 MB, but sometimes (~50% of time) curl complains with e.g.:
transfer closed with 1489665 bytes remaining to read.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the supplied nginx.conf and reproduction command, then run curl against the physical interface while checking error.log for the “task already active” alert. Compare results after removing body_filter_by_lua_block, sendfile, or aio threads, as described. Done means the configured response consistently delivers exactly 4 MB without truncation or the alert.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, lua, nginx
Domain
backend, networking, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.