hcengineering / hcengineering/huly-selfhost
v0.6.468: account/transactor log "server started on port" but never bind TCP port (504 on /_accounts/providers)
- Dominant language
- Shell
- Stars
- 3.5k
- Forks
- 469
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
Self-hosted Huly v0.6.468 (MongoDB backend). Both `account` (port 3000) and `transactor` (port 3333) containers log `server started on port ` but **never actually bind the TCP port**. Confirmed via `/proc/net/tcp`: the listening entries for ports 3000/3333 are absent even minutes after the log line appears.
Result: the front-end loads (served by `front` + `nginx`) and `/config.json` returns correct HTTPS URLs, but `GET /_accounts/providers` returns **504 Gateway Timeout** because nginx proxies to `account:3000` which never accepts connections. Creating an account is impossible.
## Environment
- Huly version: `v0.6.468` (all `hardcoreeng/*` images pinned to this tag)
- Backend: MongoDB (`mongo:7-jammy`), no CockroachDB
- Orchestrator: Dokploy (Docker Compose), on a KVM host
- Front proxy: Cloudflare -> Traefik -> nginx (Huly's bundled nginx) -> front/account/transactor
- `.env`: `HOST_ADDRESS=huly.example.dev`, `SECURE=true`, `HULY_VERSION=v0.6.468`
- MongoDB is healthy (`db.adminCommand('ping')` returns `{ ok: 1 }`), 0 collections (prepare never runs)
## Logs
```
# transactor
{"level":"info","message":"########################SplitLogger server initialized###########################"}
{"accountsUrl":"http://account:3000","level":"info","message":"starting server on","parallel":8,"port":3333}
(node:1) [DEP0040] DeprecationWarning: The `punycode` module is deprecated.
# ... then silence. No further log lines. No error. CPU stays at ~0.5%.
# account
{"level":"info","message":"########################SplitLogger account initialized###########################"}
Starting account service with brandings: {}
server started on port 3000
# ... then silence. CPU stays at ~0.06%.
```
## What I confirmed
1. **Port never opens.** `cat /proc/net/tcp` inside the transactor container shows **no entry for `0x0D05` (3333)** in state `0A` (LISTEN). Same for account (`0x0BB8` / 3000). Only an unrelated high port (e.g. 40913) on 127.0.0.1 is listening.
2. **Process is alive but idle.** `ps aux` shows `node bundle.js` as PID 1, ~80 MB RSS, CPU < 1%. Not crashed, not OOM-killed.
3. **No error logged.** No exception, no rejection, no "server error" from the `httpServer.on('error')` handler. The process just hangs after printing "starting server on".
4. **MongoDB is reachable.** `mongosh --eval "db.adminCommand('ping')"` from inside the transactor container returns `{ ok: 1 }`. Connection count in mongo logs increases.
5. **Stats availability matters earlier.** When `stats` container is not running, `account` crashes with `getaddrinfo EAI_AGAIN stats`. With `stats` up, `account` no longer crashes but still does not bind 3000.
6. **Circular dependency is not the cause.** I tried starting services in order (mongodb -> stats -> transactor -> account) with explicit `depends_on`. Same outcome: both log "started" but neither binds.
7. **`DB_PREPARE=false`** on transactor: no effect, still no bind.
8. **v0.7.432** is not a workaround: it refuses to start on MongoDB (`Migrate to CockroachDB before upgrading to v7`).
## What I tried
- Boots in dependency order with `depends_on` — no change
- `DB_PREPARE=false` on transactor — no change
- `SECURE=` empty vs `SECURE=true` — no change (mixed-content was fixed separately by setting `HOST_ADDRESS` + `SECURE=true`, but the bind problem is independent)
- Upgrading to `v0.7.432` — breaks MongoDB compatibility (requires CockroachDB)
- Restarting individual containers — they re-print "started" and hang again
- Inspected `bundle.js` (`startHttpServer2` at ~L180297): `httpServer.listen(port)` is reached only after all route/handler registration, which looks synchronous; no obvious await that could hang
## Suspected area
Something between the "starting server on" log line and `httpServer.listen(port)` in `startHttpServer2` is blocking on a promise that never resolves and never rejects. Candidates: an internal handshake with the account service (transactor side) or with the transactor websocket (account side) that runs during handler registration, creating a deadlock where each waits for the other to be listening — but neither ever reaches `listen`.
## docker-compose.yml (relevant services)
(attached — generated by Dokploy, based on the official selfhost template; `account`, `transactor`, `workspace`, `collaborator` all reference each other via `ACCOUNTS_URL` / `TRANSACTOR_URL` with no `depends_on`, but adding `depends_on` did not fix it)
## Ask
Has anyone seen "log says started, port never opens" on v0.6.468 with MongoDB? Is there a known-good image tag between v0.6.468 and v0.7 that still supports MongoDB and fixes this? Any env var that forces the HTTP server to bind before any internal handshake?
Happy to provide full `docker-compose.yml`, `.env` (redacted), and `docker logs` from every container.
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the issue with the attached Docker Compose configuration and inspect the account and transactor logs alongside bundle.js, especially startHttpServer2 around line 180297. Trace the path between the startup log and httpServer.listen(port), then verify that ports 3000 and 3333 listen and that /_accounts/providers no longer returns a 504.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose, mongodb, node.js
- Domain
- backend, databases, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100