SignalR redis perf : consider sharded pub/sub when available
- Dominant language
- C#
- Stars
- 38.4k
- Forks
- 10.9k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 290
Description
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Is your feature request related to a problem? Please describe the problem.
Short version: consider integrating with https://github.com/StackExchange/StackExchange.Redis/pull/2498/ (when deployed) to exploit `SSUBSCRIBE`/`SPUBLISH` when appropriate
Justification: overall system performance by reducing overheads in redis-cluster (no client-specific performance impact, but the reduced server load should measurably improve overall system throughput - especially in large clusters and where multiple channels are in use, since the number of server-to-server connections is super-linear vs node count)
---
SignalR on redis makes extensive use of `SUBSCRIBE` / `PUBLISH`; on redis vanilla, that is simple, but in "cluster" mode (for highly scalable or redundant environments), these operate in a simple scatter/web mode - which is to say: a subscriber can connect to any server, and a publisher can publish on any server - the server will use inter-node gossip to route the messages everywhere. This is simple and easy, but has additional overheads:
- lots of inter-server communication to get data everywhere
- possible unnecessary delays and ordering concerns
These overheads seem directly relevant to high throughput SignalR scenarios.
To reduce these overheads, Redis 7 adds [`SSUBSCRIBE`](https://redis.io/commands/ssubscribe/) and [`SPUBLISH`](https://redis.io/commands/spublish/) - the sharded variant of pub/sub - *without* chaos mode. This means that clients need to subscribe and publish using standard routing. Because of the risk of undelivered messages, and the fact that it is Redis 7 only: even though SE.Redis *already* routes using the channel, we do not default to `SSUBSCRIBE` / `SPUBLISH`.
Support for this feature is added in https://github.com/StackExchange/StackExchange.Redis/pull/2498/ ; note that you should ideally use configuration and/or feature detection (it has been added to the features API) and switch between `RedisChannel.Literal` and `RedisChannel.Sharded`.
Note also that there is a migration caveat: if migrating to enable the use of sharded, then in a hybrid app-cluster (i.e. some nodes updated, some not), the nodes using shaded vs vanilla usage might not cooperate nicely
### Describe the solution you'd like
consider using the new API when available
### Additional context
1. note that *pattern* (glob, wildcard, etc) channel subscriptions cannot be sharded; they *must* use the web/chaos mode
2. note that `SPUBLISH`/`SSUBSCRIBE` require either a server capability test or an enable/disable toggle (either server-version based, or by attempting `SPUBLISH {random guid} {whatever}` and observing `-ERR unknown command 'spublish'`)
3. note that `SPUBLISH`/`SSUBSCRIBE` are fine to use on standalone (non-cluster) redis
4. note that `SPUBLISH` and `PUBLISH` are effectively isolated; a client connecting via `SUBSCRIBE foo` *will not* get a message published via `SPUBLISH foo bar`, etc; this means that nodes in an application must agree on the strategy being used
Side note: since this requires a lib bump, you might *also* want to consider whether enabling RESP3 is appropriate; RESP3 halves the number of sockets required for pub/sub environments; see [RESP3 and StackExchange.Redis](https://stackexchange.github.io/StackExchange.Redis/Resp3) for more info including necessary considerations
Contributor guide
Assessment
This issue has not been assessed yet.