getsentry / getsentry/self-hosted

Support clustered Redis

Open
#1,923 10 comments 12 reactions 1 assignee Claimed by @aldy505 View on GitHub
Improvement
Dominant language
Shell
Stars
9.6k
Forks
2k
Avg merge
1d 5h
Merged PRs (30d)
15

Description

## Important Details

How are you running Sentry?

* [x] On-Premise docker [Version 8.21]
* [ ] Saas (sentry.io)
* [ ] Other [briefly describe your environment]

## Description

I'm trying to use Sentry with Redis provided by AWS ElastiCache. I get a ton of errors in the logs and similar output when I try to do `sentry queues list`.

```
Sep 24 21:56:49 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: ResponseError: MOVED 5945 172.31.103.250:6379
Sep 24 21:56:49 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: 21:56:49 [ERROR] sentry.errors: Unable to incr internal metric
Sep 24 21:56:49 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: ResponseError: MOVED 7359 172.31.103.250:6379
Sep 24 21:56:49 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: 21:56:49 [ERROR] sentry.errors: Unable to incr internal metric
Sep 24 21:57:19 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: ResponseError: MOVED 5945 172.31.103.250:6379
Sep 24 21:57:19 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: 21:57:19 [ERROR] sentry.errors: Unable to incr internal metric
Sep 24 21:57:19 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: ResponseError: MOVED 3474 172.31.103.59:6379
Sep 24 21:57:19 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: 21:57:19 [ERROR] sentry.errors: Unable to incr internal metric
Sep 24 21:57:19 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: ResponseError: MOVED 3474 172.31.103.59:6379
Sep 24 21:57:19 ecs-cluster-production-reserved-on-demand134 docker/sentry/log/f6e3b6c8feb5[4501]: 21:57:19 [ERROR] sentry.errors: Unable to incr internal metric
```

## Steps to Reproduce

1. create redis in ElastiCache without encryption but with clustering enabled
2. run Sentry in an ECS task
3. look at logs

Also:

1. the ECS task is listening.

```
root@ecs-cluster-production-reserved-on-demand134:/mnt/var/log/docker/sentry# docker ps | grep sentry
f6e3b6c8feb5 docker-registry.blueshift.vpc/sentry:8.21 "/entrypoint.sh ru..." 15 minutes ago Up 15 minutes 0.0.0.0:44225->9000/tcp ecs-sentry-5-sentry-d085b6d5cbffc9c75400
root@ecs-cluster-production-reserved-on-demand134:/mnt/var/log/docker/sentry# curl localhost:44225/_health/ ; echo
ok
```

2. I tried `sentry repair` and it didn't seem to help.
3. I see getsentry/sentry#3745 but since that was 3 years ago opinions and code could have changed.

### What you expected to happen

Sentry would start and talk to redis.

### Possible Solution

I'm guessing that moving to non-clustered redis might fix it, but then we're less reliable and scalable.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.