redis / redis/node-redis

Error, which is returned from retry_strategy remains uncaught

Open
#1,202 23 comments 21 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Pending Author Input
Dominant language
TypeScript
Stars
17.6k
Forks
2k
Avg merge
2d 3h
Merged PRs (30d)
40

Description

I have the code below and try to imitate dropped connection using iptables -A OUTPUT -p tcp --dport 6379 -j REJECT.

self.client = redis.createClient(self.cfg.port, self.cfg.host, {
    retry_strategy: function (options) {
        console.log('retry strategy check');
        console.log(options);
        if (options.error) {
            if (options.error.code === 'ECONNREFUSED') {
                // End reconnecting on a specific error and flush all commands with a individual error
                return new Error('The server refused the connection');
            }
            if (options.error.code === 'ECONNRESET') {
                return new Error('The server reset the connection');
            }
            if (options.error.code === 'ETIMEDOUT') {
                return new Error('The server timeouted the connection');
            }
        }
        if (options.total_retry_time > 1000 * 60 * 60) {
            // End reconnecting after a specific timeout and flush all commands with a individual error
            return new Error('Retry time exhausted');
        }
        if (options.attempt > 5) {
            // End reconnecting with built in error
            return new Error('Retry attempts ended');
        }
        // reconnect after
        return 1000;
    }
});
self.client.on('ready', function () {
    log.trace('Redis client: ready');
});
self.client.on('connect', function () {
    log.trace('Redis client: connect');
});
self.client.on('reconnecting', function () {
    log.trace('Redis client: reconnecting');
});
self.client.on('error', function (err) {
    log.error({err: err}, 'Listener.redis.client error: %s', err);
    process.exit(1);
});
self.client.on('end', function () {
    log.trace('Redis client: end');
});
self.client.on('warning', function () {
    log.trace('Redis client: warning');
});

It is supposed that all redis errors are emitted in error event. But here is what i've got in console output:

21:00:14.666Z TRACE script: Redis client: connect
21:00:14.695Z TRACE script: Redis client: ready
21:10:23.837Z TRACE script: Redis client: end
retry strategy check
{ attempt: 1,
error: { [Error: Redis connection to redis.callision.info:6379 failed - read ECONNRESET] code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' },
total_retry_time: 0,
times_connected: 1 }

/node_modules/q/q.js:155
throw e;
^
AbortError: Stream connection ended and command aborted. It might have been processed.
at RedisClient.flush_and_error (/node_modules/redis/index.js:350:23)
at RedisClient.connection_gone (/node_modules/redis/index.js:612:18)
at RedisClient.on_error (/node_modules/redis/index.js:398:10)
at Socket. (/node_modules/redis/index.js:272:14)
at emitOne (events.js:90:13)
at Socket.emit (events.js:182:7)
at emitErrorNT (net.js:1255:8)
at nextTickCallbackWith2Args (node.js:474:9)
at process._tickCallback (node.js:388:17)

And as a question: Why it takes about 10 minutes to detect that connection is gone? Is there any way to raise an error in case of no response within 10 seconds? May be any option like response_timeout etc.

  • Version: node_redis v.2.6.5 and Redis 3.0.7
  • Platform: Node.js v5.5.0 on Ubuntu 14.04.4 LTS
  • Description: error from retry_strategy remains uncaught

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the connection and retry handling in node_modules/redis/index.js, especially the stack locations in flush_and_error, connection_gone, and on_error, and compare them with the retry_strategy and error-listener behavior described here. Reproduce the ECONNRESET case and investigate the delayed detection and missing propagated error; done means the behavior is covered by tests and the documented timeout or error semantics match the issue’s expectations.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs, redis
Domain
backend, databases, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.