openresty / openresty/lua-resty-upload

lua tcp socket read timed out

Open
#35 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Lua
Stars
414
Forks
110
Avg merge
2h 43m
Merged PRs (30d)
1

Description

hi,I use lua-resty-upload to handle big file upload, I use an extension of this version which have the part-split upload function, repos link, when the last part upload complete, the nginx error log sometimes show this error:
lua tcp socket read timed out
and did not merge all the part to one file,please give me a help!thx!
this is upload.lua script below:

local upload = require "resty.upload"
local cjson = require "cjson"
local get = ngx.req.get_uri_args()
local chunk_size = 4096
local file_upload_count = 0
local speed = 0
local post = {}
local index
local file
local upload_path = "/tmp/" -- 上传路径

--merge chunks
local function mergeChunks(chunks, name)
    local file = io.open(upload_path .. name, "w+")
    for i = 0, chunks - 1 do
        local chunk_path = upload_path .. name .. "." .. i
        local chunk_file = io.open(chunk_path)
        while true do
            local bytes = chunk_file:read(4096)
            if not bytes then
                break
            end
            file:write(bytes) 
        end
        chunk_file:close()
        os.remove(chunk_path)
    end
    file:close()
end

-- rate limit
if get.speed then
    speed = math.ceil(get.speed * 1024 / chunk_size)
end

local form, err = upload:new(chunk_size)
if not form then
    ngx.say("no zuo no die.")
    ngx.exit(200)
end

form:set_timeout(1000) -- one second timeout

while true do
    local typ, res, err = form:read()
    if not typ then
        ngx.say("failed to read: ", err)
        return
    end

    if typ == "header" then
        if res[1] == "Content-Disposition" then
            local filename = ngx.re.match(res[2], '(.+)filename="(.+)"(.*)')

            if filename then
                filename = filename[2]

                -- part upload
                if post["chunks"] then
                    filename = filename .. "." .. post["chunk"]
                end

                file = io.open(upload_path .. filename, "w+")
                if not file then
                    ngx.say("failed to open file ")
                    return
                end
            else
                local name = ngx.re.match(res[2], '(.+)name="(.+)"(.*)')
                if name[2] then
                    index = name[2]
                end    
            end
        end
    elseif typ == "body" then
        if file then
            -- record upload times
            file_upload_count = file_upload_count + 1

            -- rate limit
            if speed ~= 0 and file_upload_count % speed == 0 then
                ngx.sleep(1)
            end

            -- write file
            file:write(res)
        else
            if index then
                if post[index] == nil then
                    post[index] = res
                else
                    post[index] = post[index] .. res
                end
            end
        end
    elseif typ == "part_end" then
        if file then
            file:close()
            file = nil

            if post["chunks"] and post["chunk"] then
                if tonumber(post["chunks"]) - tonumber(post["chunk"]) == 1 then
                    mergeChunks(tonumber(post["chunks"]), post["name"])
                end
            end

            ngx.header["Content-Type"] = 'application/json'
            ngx.say(cjson.encode({code=1}))
        end
        if index then
            index = nil
        end
    elseif typ == "eof" then
        break
    else

    end

end

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked lua/upload.lua script, especially form:set_timeout(1000), form:read(), and mergeChunks(). Trace the upload sequence around the read timeout and final chunk handling; done means the cause is reproducible and uploads complete with all chunks merged without the timeout error.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.