redis / redis/node-redis

Enable redis client to be paused and resumed using underlying stream's pause and resume

Open
#1,500 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
17.6k
Forks
2k
Avg merge
2d 3h
Merged PRs (30d)
40

Description

Issue

node-redis doesn't support the ability to pause and resume the data coming into it. That means if a node process is taking data off redis (for example pubsub) and storing it in another location that has less operations per second the requests from redis will pile up inside nodejs or worse the requests will exceed the limits of the upstream location and requests will begin to fail and drop at that location. Nodejs applications need the ability to manage their own resource consumption. Currently with redis calling callbacks for pubsub events there is no way to backpressure on the redis client and back to the tcp connection. See node guide to backpressure

I am thinking a simple function mirroring the underlying function would be good

const redis = require("redis");
const client = redis.createClient();
client.pause();
client.resume();

I would like to build a nodejs stream on top of redis pubsub and/or redis stream. I think the best way to do this would be to support pause and resume on the client.

The stream would wrap around the client something like

const redis = require("redis");
const redisStream = require("redis-stream");
const { pipeline } = require('stream');
const subscriber = redis.createClient();
subscriber.subscribe("a channel");
const subscriberStream  = redisStream.createReadStream(subscriber, {include_connect: true});

pipeline(subscriberStream, process.stdout)

Environment
  • Node.js Version: 12.16.1
  • Redis Version: 5.0.3
  • Platform: 10.14.6

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the linked Node.js backpressure guide, then start at the Redis client API and the underlying stream mentioned in the issue. Determine how pause and resume should affect incoming pubsub data and the TCP connection, and verify the example client.pause() and client.resume() usage behaves as requested.

Written by the indexing model from the issue text.

Assessment

Tech stack
nodejs, redis, typescript
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.