request to aws kinesis getRecords endpoint aborted by error 'closed'

hi, I'm author of lua-aws, lua implementation of aws API which supports openresty. thank you for creating resty-cli, it helps to check lua-aws' behavior in openresty!

this issue may relate with both resty-cli and lua-nginx-module or nginx itself. if here is not proper place to report the issue, please kindly show me pointer.

problem description

I tried to send request to AWS kinesis getRecord endpoint(https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetRecords.html) with using lua-resty-http(https://github.com/ledgetech/lua-resty-http) like https://github.com/umegaya/lua-aws/blob/master/lua-aws/engines/http/lua-resty-http.lua, which I think no special usage there.

then lua-aws got following error.

kinesis:v[2013-12-02]:GetRecords:error:./lua-aws/engines/http/lua-resty-http.lua:27: closed

with further investigation, it seems to caused by failure of ngx.socket.tcp.receive(https://github.com/ledgetech/lua-resty-http/blob/master/lib/resty/http.lua#L354), likely comes from lua-nginx-module(https://github.com/openresty/lua-nginx-module/blob/master/src/ngx_http_lua_socket_tcp.c#L2156)

some insight

I don't think it is a problem of our usage of lua-resty-http and lua-nginx-module because other kinesis endpoint like putRecords(https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecords.html) works perfectly.

also, this does not seems to be kinesis' getRecords problem, because if we use luasocket to communicate with kinesis getRecords endpoint, it works as expected.

I guess nginx or lua-nginx-module detect connection close a bit earlier (maybe because of getRecords endpoint behavior which does not affect luasocket implementation), than getRecords API expected for some reason.

any idea why this problem happens/workaround for the issue?

regards,

Start by reproducing the AWS Kinesis GetRecords failure through lua-resty-http, comparing it with the working PutRecords request and the LuaSocket behavior. Read lua-aws/engines/http/lua-resty-http.lua at line 27, resty/http.lua around line 354, and ngx_http_lua_socket_tcp.c around line 2156. Done means identifying which layer closes the connection and documenting a confirmed workaround or fix.