self-hosted cymbal container failed to connect redis
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 6h 51m
- Merged PRs (30d)
- 232
Description
Bug Description
Hello, to solve the problem of error tracking on self-hosted (https://github.com/posthog/posthog/issues/38918), I tried to start the cymbal container with this PR(https://github.com/PostHog/posthog/pull/41841).
The cymbal image's hash version is ghcr.io/posthog/posthog/cymbal@sha256:7f1afe72268f150310a303bdfc8f17db27709fcd134d12b1de8a8391ef3c67ef
(extracted from the following output:
dev@poc-vm-02:~/posthog-new/posthog-master$ docker image inspect 9caca62a58e4
[
{
"Id": "sha256:9caca62a58e42dd7566417120aec90e70a683f3183684c2adaa2cfd3ffb54916",
"RepoTags": [
"ghcr.io/posthog/posthog/cymbal:master"
],
"RepoDigests": [
"ghcr.io/posthog/posthog/cymbal@sha256:7f1afe72268f150310a303bdfc8f17db27709fcd134d12b1de8a8391ef3c67ef"
],
"Parent": "",
"Comment": "buildkit.dockerfile.v0",
"Created": "2026-02-09T18:01:44.135893801Z",
"DockerVersion": "",
"Author": "",
......
)
But when the cymbal container failed to start with the following error:
2026-02-11T02:20:52.842359Z INFO cymbal: Starting up...
2026-02-11T02:20:52.842449Z INFO common_continuous_profiling: Continuous profiling is disabled
2026-02-11T02:20:52.842465Z WARN cymbal: Posthog client disabled
2026-02-11T02:20:52.851722Z WARN cymbal::config: Falling back to building config from explicit environment variables
thread 'main' (7) panicked at cymbal/src/main.rs:57:67:
called `Result::unwrap()` on an `Err` value: RedisError(Redis(Connection refused (os error 111)))
stack backtrace:
0: 0x5630c93debf2 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::haa87a551a4affa55
1: 0x5630c93f284f - core::fmt::write::h80461e1e45e4fdd2
2: 0x5630c93ab0e1 - std::io::Write::write_fmt::h6e6c69b2d6337d9b
3: 0x5630c93b7a42 - std::sys::backtrace::BacktraceLock::print::hf67a46baa621998e
4: 0x5630c93bd0ff - std::panicking::default_hook::{{closure}}::h391aa815d5e47ec8
5: 0x5630c93bcf59 - std::panicking::default_hook::hd6fdcf2489bb807d
6: 0x5630c93bd835 - std::panicking::panic_with_hook::h185ddfb86bf14d73
7: 0x5630c93bd61a - std::panicking::panic_handler::{{closure}}::had89ddd01b6112c9
8: 0x5630c93b7b79 - std::sys::backtrace::__rust_end_short_backtrace::h5d0fc36eef7265ea
9: 0x5630c939e62d - __rustc[eb8946e36839644a]::rust_begin_unwind
10: 0x5630c93fdd50 - core::panicking::panic_fmt::h92c8e5abe71dd8d1
11: 0x5630c93fcd56 - core::result::unwrap_failed::h0501379eaec3e720
12: 0x5630c799c22b - cymbal::main::{{closure}}::ha261d8a87d1404ed
13: 0x5630c7991b84 - tokio::runtime::park::CachedParkThread::block_on::h3c9b65ff98ab6ae1
14: 0x5630c7729cec - tokio::runtime::runtime::Runtime::block_on::h1736034a67f4e933
15: 0x5630c77dae52 - cymbal::main::h3f587d706d633b2d
16: 0x5630c7945353 - std::sys::backtrace::__rust_begin_short_backtrace::hcf6f41e831909d84
17: 0x5630c7879579 - std::rt::lang_start::{{closure}}::h92179c0834e0ea86
18: 0x5630c93acc10 - std::rt::lang_start_internal::h6ba36b077a531782
19: 0x5630c77e6b55 - main
20: 0x7f2a128a724a - <unknown>
21: 0x7f2a128a7305 - __libc_start_main
22: 0x5630c76ad121 - _start
23: 0x0 - <unknown>
And the following is how I configure the redis in env variables:
cymbal:
image: ghcr.io/posthog/posthog/cymbal:master
build:
context: rust/
args:
BIN: cymbal
restart: on-failure
environment:
REDIS_URL: "redis://109.144.5.242:6381"
PERSONS_URL: "postgres://posthog:posthog@db:5432/posthog"
DATABASE_URL: "postgres://posthog:posthog@db:5432/posthog" #'postgres://posthog:posthog@109.144.6.162:5432/posthog' #'postgres://posthog:posthog@db:5432/posthog'
KAFKA_HOSTS: 'kafka:9092'
#RUST_BACKTRACE: "1"
RUST_BACKTRACE: "full"
depends_on:
db:
condition: service_healthy
kafka:
condition: service_started
There is no error with redis because I can connect it as follows:
I have no name!@redis-cluster-0:/$ redis-cli -u redis://109.144.5.242:6381
109.144.5.242:6381> dbsize
(integer) 1506
109.144.5.242:6381>
Is there anything I can do to solve this? thanks!
Debug info
- [√ ] PostHog Hobby self-hosted with `docker compose`, version/commit: [build from https://github.com/PostHog/posthog/commit/7866528acd998bbb51c8afcc24749aeea0a06e2d]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at cymbal/src/main.rs:57 and review the cymbal service environment in the shown Docker Compose configuration, especially REDIS_URL. Compare the container's Redis connection context with the successful redis-cli test. Done means cymbal starts without panicking and connects to Redis successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker-compose, redis, rust
- Domain
- backend, databases, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100