openresty / openresty/lua-resty-redis

Redis MULTI, connection pool and WS problem

Open
#176 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Lua
Stars
2k
Forks
447
Avg merge
2h 24m
Merged PRs (30d)
3

Description

Hi,

We came across an interesting situation when running our app which roughly has:

  • an event loop that keeps calling recv_frame. On close or error we call set_keepalive to keep the redis connection in a pool.
  • a function (called from within the event loop) that uses redis MULTI - EXEC to run multiple statements in a "transaction".

The issue is that sometimes the connection that get added to the pool is 'in the middle of a transaction' (we know this because, when the redis connection is reused a call to incrby for example returns QUEUED instead of the int).

It's a bit like as if the lua function that does MULTI - EXEC is preempted by the event loop before it can finish.

The redis function is very simple, just a MULTI...few INCR....an EXEC.

Maybe it's just me not understanding how lua-nginx concurrency model but I assumed that any lua function would be executed completely before any other function (i.e. the event loop cycle) could continue. Is that not the case?

The only other thing that I can think of is if the EXEC had to fail and leave the transaction still active. Is that a possibility? How to overcome that?

Thanks :)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the event-loop entry point around recv_frame and the connection-pool handoff through set_keepalive, then review the MULTI/EXEC sequence and its responses. Reproduce reuse of a connection during or after a transaction and determine whether the transaction can remain active; done means the failure path and a reliable way to prevent or detect it are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua, redis
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.