Hobby self-hosted: docker-compose.base.yml is missing two ENVs that error tracking needs end-to-end

Summary

On a fresh Hobby self-hosted deployment (latest master), $exception events go in but never come out: events table stays at 0, error_tracking_* tables stay at 0, the Error Tracking UI shows "No Exception events have been detected!" forever.

The cymbal and ingestion-error-tracking containers are both Up but completely idle.

The reason is that docker-compose.base.yml doesn't pass two environment variables that the rest of the stack assumes are set. Both are set explicitly in the bin/start-rust-service and bin/start local-dev launchers, but those launchers aren't used in the docker-compose path.

Adding two lines to docker-compose.base.yml fixes the whole pipeline.

---

Environment

• Deployment: docker-compose.hobby.yml extending docker-compose.base.yml, latest master
• capture image: ghcr.io/posthog/posthog/capture:master (built at commit 8d12f459aa312f7ff967edf5535607494d659e71)
• cymbal: healthy, listening on 0.0.0.0:3302
• ingestion-error-tracking: healthy, "All systems go in 134ms"
• All Kafka topics auto-created (error_tracking_events AND ingestion-errortracking-main)
• Project setting autocapture_exceptions_opt_in = True

---

Root cause (verified end-to-end)

Defect 1: capture writes to the wrong topic

rust/capture/src/config.rs defaults kafka_error_tracking_topic to "error_tracking_events":

#[envconfig(default = "error_tracking_events")]
pub kafka_error_tracking_topic: String,

But the consumer side (nodejs/src/config/kafka-topics.ts) consumes a different topic:

KAFKA_ERROR_TRACKING_INGESTION = `${prefix}ingestion-errortracking-main${suffix}`

bin/start-rust-service knows about this and explicitly bridges the gap:

# bin/start-rust-service:52
export KAFKA_ERROR_TRACKING_TOPIC=${KAFKA_ERROR_TRACKING_TOPIC:-ingestion-errortracking-main}

But docker-compose.base.yml's capture service doesn't:

capture:
    environment:
        ADDRESS: '0.0.0.0:3000'
        KAFKA_TOPIC: 'events_plugin_ingestion'
        KAFKA_HOSTS: 'kafka:9092'
        REDIS_URL: 'redis://redis7:6379/'
        CAPTURE_MODE: events
        RUST_LOG: 'info,rdkafka=warn'
        # ← no KAFKA_ERROR_TRACKING_TOPIC

So the rust capture service routes $exception events to a topic (error_tracking_events) that nothing in the stack consumes, while ingestion-errortracking-main stays at LOG-END-OFFSET=0 forever.

Defect 2: ingestion-error-tracking calls cymbal at the wrong host

nodejs/src/ingestion/error-tracking/config.ts defaults ERROR_TRACKING_CYMBAL_BASE_URL to http://localhost:3302:

ERROR_TRACKING_CYMBAL_BASE_URL: 'http://localhost:3302',

That's correct for monolithic local dev (bin/start sets it explicitly to http://lvh.me:3302):

# bin/start
# Error tracking — Node.js ingestion consumer needs to reach Cymbal (Rust) on the host
# Using lvh.me instead of localhost for cross-platform compatibility
export ERROR_TRACKING_CYMBAL_BASE_URL=${ERROR_TRACKING_CYMBAL_BASE_URL:-http://lvh.me:3302}

But in docker-compose, cymbal is a separate container and Compose's network DNS exposes it as cymbal:3302, not localhost:3302. docker-compose.base.yml doesn't set this either:

ingestion-error-tracking:
    command: node nodejs/dist/index.js
    restart: on-failure
    environment:
        PLUGIN_SERVER_MODE: 'ingestion-errortracking'
        DATABASE_URL: '...'
        KAFKA_HOSTS: 'kafka:9092'
        REDIS_URL: 'redis://redis7:6379/'
        # ← no ERROR_TRACKING_CYMBAL_BASE_URL

This silently breaks anyway because, with Defect 1 in place, the consumer never receives a message and never tries to call cymbal — so the misconfiguration goes unnoticed until you fix Defect 1, and then it surfaces as a graceful-shutdown loop.

---

Reproduction

1. git clone posthog && cd posthog && docker compose -f docker-compose.hobby.yml up -d
2. Wait for posthog-cymbal-1 and posthog-ingestion-error-tracking-1 to be Up.
3. From any host:

   curl -X POST https://your-posthog.example.com/capture/ \
     -H 'Content-Type: application/json' \
     -d '{
       "api_key":"phc_xxx",
       "event":"$exception",
       "distinct_id":"repro_user",
       "properties":{
         "$exception_list":[{"type":"RuntimeError","value":"hello","mechanism":{"type":"generic","handled":true}}]
       }
     }'
   # → {"status":"Ok"} HTTP 200
   

4. Wait 30s for ingestion.
5. Open the Error Tracking page in the UI. Empty.
6. Verify the Kafka topic that the consumer is reading:

   docker exec posthog-kafka-1 rpk topic describe ingestion-errortracking-main -p
   # PARTITION  ... LOG-END-OFFSET
   # 0          ... 0
   

7. Now check the topic that capture actually wrote to (default):

   docker exec posthog-kafka-1 rpk topic describe error_tracking_events -p
   # PARTITION  ... LOG-END-OFFSET
   # 0          ... 1
   

   The events are stuck on a topic nothing consumes.

---

Fix (proposed PR — minimal, two lines)

--- a/docker-compose.base.yml
+++ b/docker-compose.base.yml
@@ capture:
     environment:
         ADDRESS: '0.0.0.0:3000'
         KAFKA_TOPIC: 'events_plugin_ingestion'
+        KAFKA_ERROR_TRACKING_TOPIC: 'ingestion-errortracking-main'
         KAFKA_HOSTS: 'kafka:9092'
         REDIS_URL: 'redis://redis7:6379/'
         CAPTURE_MODE: events
         RUST_LOG: 'info,rdkafka=warn'

@@ ingestion-error-tracking:
     environment:
         PLUGIN_SERVER_MODE: 'ingestion-errortracking'
         DATABASE_URL: 'postgres://posthog:posthog@db:5432/posthog'
         KAFKA_HOSTS: 'kafka:9092'
         REDIS_URL: 'redis://redis7:6379/'
+        ERROR_TRACKING_CYMBAL_BASE_URL: 'http://cymbal:3302'
         CDP_REDIS_HOST: redis7
         ENCRYPTION_SALT_KEYS: '00beef0000beef0000beef0000beef00'

After applying these two ENVs and docker compose up -d capture ingestion-error-tracking:

• Kafka topic ingestion-errortracking-main starts receiving messages from capture.
• The consumer joins the ingestion-errortracking group, becomes STATE: Stable, MEMBERS: 1.
• cymbal receives fingerprint computation requests at http://cymbal:3302.
• Events flow into events and the Error Tracking issue tables in ClickHouse.
• The "No Exception events have been detected!" banner goes away.

I verified this end-to-end on my deployment.

---

Bonus: better empty-state messaging

The current empty-state banner blames the SDK:

"No Exception events have been detected!
To use the Error tracking product, please enable exception capture within the PostHog SDK (otherwise it'll be a little empty!)"

But the SDK is fine in this case — the issue is purely server-side. We spent significant time chasing the SDK side before figuring out the routing was broken in docker-compose.base.yml. A more accurate message when the deployment is server-side broken:

"No Exception events have been detected!

• SDKs not yet configured? See the docs.
• Already sending? On self-hosted, verify KAFKA_ERROR_TRACKING_TOPIC and ERROR_TRACKING_CYMBAL_BASE_URL are set on capture and ingestion-error-tracking respectively."

Or, if there's a control plane that can probe the consumer-group state, simply:

"Capture is running but the ingestion-errortracking-main topic has 0 messages. This usually means capture is missing KAFKA_ERROR_TRACKING_TOPIC=ingestion-errortracking-main."

---

What I can contribute

I have a reproducible setup against master and have already verified the two-line patch fixes Hobby self-hosted end-to-end. Happy to send the PR if useful.

Start in docker-compose.base.yml and compare the capture and ingestion-error-tracking environment blocks with bin/start-rust-service and bin/start. Recreate the Hobby deployment, send a $exception event, and verify that ingestion-errortracking-main receives it, the consumer reaches cymbal, and Error Tracking tables are populated.