Enable redis client to be paused and resumed using underlying stream's pause and resume
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 17.6k
- Forks
- 2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 40
Description
Issue
node-redis doesn't support the ability to pause and resume the data coming into it. That means if a node process is taking data off redis (for example pubsub) and storing it in another location that has less operations per second the requests from redis will pile up inside nodejs or worse the requests will exceed the limits of the upstream location and requests will begin to fail and drop at that location. Nodejs applications need the ability to manage their own resource consumption. Currently with redis calling callbacks for pubsub events there is no way to backpressure on the redis client and back to the tcp connection. See node guide to backpressure
I am thinking a simple function mirroring the underlying function would be good
const redis = require("redis");
const client = redis.createClient();
client.pause();
client.resume();
I would like to build a nodejs stream on top of redis pubsub and/or redis stream. I think the best way to do this would be to support pause and resume on the client.
The stream would wrap around the client something like
const redis = require("redis");
const redisStream = require("redis-stream");
const { pipeline } = require('stream');
const subscriber = redis.createClient();
subscriber.subscribe("a channel");
const subscriberStream = redisStream.createReadStream(subscriber, {include_connect: true});
pipeline(subscriberStream, process.stdout)
Environment
- Node.js Version:
12.16.1
- Redis Version:
5.0.3
- Platform:
10.14.6
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read the linked Node.js backpressure guide, then start at the Redis client API and the underlying stream mentioned in the issue. Determine how pause and resume should affect incoming pubsub data and the TCP connection, and verify the example client.pause() and client.resume() usage behaves as requested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nodejs, redis, typescript
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100