Discussion: Blocking operations

We've gotten asks for this in things like #1474, #1593, and #1675.

The general ask is support for blocking operations like WAIT but also others like XREAD. The general problem here is with a multiplexed connection, all commands behind the blocking command are waiting on it and an indefinite time period, and so this is a very easy footgun to wield.

Proposals

One idea has been something like an IBlockingDatabase or similar where a user would have to explicitly cast to an interface to use it, but the more I think about it maybe that's not the correct abstraction. For one: these are per-client, per database, per-stream, and other things - so there would be multiple. But overall, the effect is per connection (or specifically in our internal nomenclature, per bridge).

What if instead this was on ConfigurationOptions to allow a blocking command. We could have one bool there that allows that multiplexer to perform blocking operations. Any attempt to call a blocking operation without this enabled would throw an informative error to the user. As these would be new commands or overloads not in play today, there wouldn't be any API break. This error message could point to our GitHub Pages help informing people about what this means and how to be careful.

Connection pooling I have worries about with respect to ordering but I believe these use cases to be completely orthogonal to ordering, so if we get pooling in this same options approach could be used to spin up n bridges or whatever approach we take there. Today we wouldn't want to do that because many have connection limits and don't want the subscriber one either (which isn't something we can disable today).

I'm curious what others thing of this: basically an option that says "yes, I accept the disclaimer" to allow blocking commands?

Downstream problems

Of note though, there is a downstream delineation of impact here because we're on sockets that have TTLs. For something like XREAD we could have an IAsyncEmulerable API with steam results streaming out, but internally we could say GET X items every N ms (e.g. XREAD COUNT 50 BLOCK 200), and only block for that long, then try again in a while loop, basically asynchronously blocking that multiplexer - this is fairly straightforward if the N ms is under 1,000.

Beyond that 1000 ms, we run into pile-up problems with heartbeats keeping the connection alive. The problem set is basically < 1000 ms and > 1000 ms, with the latter being "up to indefinite". Indefinite is a problem I don't have a solution for because we know that many environments including paid hosting will drop the socket if there's no data being sent for too long a time period. Socket TTLs of the system is not something we can readily work around, to my knowledge. We could send noise that piles up for execution behind the WAIT to the Redis server from the client, but nothing coming back on any defined time period is ensured as the timeout is indefinite and the pile up itself is also unbounded.

We could back off if we know the pending/current command is a block (with a flag?) and only KeepAlive once a minute or something to lower the load, but again that only solves socket death in one direction. But for limited blocks that have a millisecond duration specifier like XREAD, this isn't something we have to solve to unlock. Even without connection pooling or unbounded numbers of bridges, users could create a dedicated multiplexer with this option to read a stream with the proposal above. There could be 10 other problems with this, just wanted to get the discussion into a single issue!

cc @mgravell @philon-msft @davidfowl

No implementation files or tests are named. Start by reviewing issues #1474, #1593, and #1675, then examine the blocking-command proposals involving ConfigurationOptions, multiplexed connections, and socket TTLs. Done is not defined yet: the project needs an agreed design for safely supporting commands such as WAIT and XREAD before implementation can begin.