PostHog / PostHog/posthog

Hobby self-hosted: NGINX Unit serves large /static/ chunks ~100x slower than Caddy file_server (blank page after deploy)

Open
#58,150 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Bug Description

After running bin/deploy-hobby and waiting for migrations, the deployed instance returns HTTP 200 on /_health but the SPA shows a blank page for users on networks where per-stream throughput to the host is limited (typical for cross-region access, capped uplinks, or shared connections).

The root cause is how /static/*.js chunks are served. The default Caddyfile reverse-proxies /static/* to the web container, where NGINX Unit serves the file via the Django app. Two large entry chunks each weigh 8+ MB uncompressed:

File Raw size
staticfiles/chunk-PHKJLKH7.js (hash will vary per build) ~8.5 MB
staticfiles/chunk-I2FYEAVV.js (hash will vary per build) ~8.2 MB

Tests against the same instance show a ~100× difference between the two serving paths:

Path Single-file download time (8.5 MB chunk)
Caddy → reverse_proxy web:8000 (NGINX Unit) 30+ s, frequently timing out
Caddy file_server direct, precompressed br gzip (2.1 MB after brotli) 0.27 s

Because these chunks are required for the React entry to mount, the SPA never renders until they finish — observed as a blank page that lasts 60–90 seconds (or never resolves) on slow links.

How to reproduce

  1. Deploy via bin/deploy-hobby latest <domain> on a fresh VPS.
  2. From a client where the path to the host caps single-stream throughput (e.g., cross-continent, mobile network, shared office uplink), open the deployed URL.
  3. DevTools → Network: observe the two largest /static/chunk-*.js files (~8 MB each) hanging for tens of seconds. React #root stays empty.

Workaround (verified)

  1. Pre-compress JS/CSS in the web image:
    find /code/staticfiles -maxdepth 1 \( -name "*.js" -o -name "*.css" \) \
         ! -name "*.br" ! -name "*.gz" ! -name "*.map" -print0 \
      | xargs -0 -P4 -I{} brotli -q 6 -k {}
    
  2. Make staticfiles available to the proxy container (volume mount or docker cp).
  3. In the Caddyfile, before the catch-all handle { reverse_proxy web:8000 }:
    handle /static/* {
        uri strip_prefix /static
        root * /usr/share/caddy/staticfiles
        file_server { precompressed br gzip }
        header Cache-Control "public, max-age=31536000, immutable"
    }
    
  4. caddy reload. Same chunks now serve at line speed; React renders in seconds.

Suggested fix

  • Either: have the official image emit *.br and *.gz alongside every staticfiles/*.{js,css} at build time, and update docker-compose.hobby.yml so the proxy container can file_server precompressed directly; or
  • Configure NGINX Unit to do Accept-Encoding-aware static serving (Unit ≥ 1.30 supports this) so the reverse-proxy path is no longer the bottleneck.

The current default makes hobby deployments effectively unusable for any audience whose network can't sustain ~25 MB of single-stream JS within typical browser stream timeouts.

Debug info

  • PostHog Cloud
  • PostHog Hobby self-hosted with docker compose, image: posthog/posthog:latest (commit 2622588f5c)
  • PostHog self-hosted with Kubernetes
  • Compose: docker-compose.hobby.yml + docker-compose.base.yml, generated by bin/deploy-hobby
  • Proxy: stock caddy image, configuration generated from CADDYFILE env var
  • Host: Ubuntu 22.04 cloud VPS, 4 vCPU / 16 GB RAM

Thanks for maintaining PostHog and the deploy-hobby script — happy to test patches.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing bin/deploy-hobby, docker-compose.hobby.yml, docker-compose.base.yml, and the CADDYFILE environment configuration. Inspect how staticfiles are produced in the web image and made available to the proxy, then reproduce the large chunk download behavior. Done means hobby deployments serve the large JavaScript chunks efficiently and the SPA renders normally on constrained links.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, docker, docker-compose, javascript, react
Domain
devops, infrastructure, performance, web-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.