Dokploy / Dokploy/dokploy

Traefik backend connections can outlive Swarm task membership during rolling updates

Open
#5,281 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs-triage🔍
Dominant language
TypeScript
Stars
37.4k
Forks
3k
Avg merge
1d 3h
Merged PRs (30d)
73

Description

To Reproduce

Dokploy generates one Traefik backend per domain, pointing to the Swarm service name:

servers:

  • url: "http://myapp:3000"

See createServiceConfig.

Traefik keeps idle connections to that address. Go’s HTTP transport pools them by dial address rather than the resolved task IP. Updating the VIP or DNS membership therefore affects new connections, but not a connection already associated with a task being replaced.

Here is the current failure mode visible in the current flow that can replicate the issue:

  1. Deploy an application with at least two replicas and a response header identifying its build.
  2. Send steady HTTP/1.1 keep-alive traffic through its Dokploy-generated domain.
  3. Deploy a new build using start-first.
  4. Watch the identifying header and connection errors throughout the replacement.
Current vs. Expected behavior

Failure/bug: once Swarm removes a task from service membership, later requests stop reaching it.

An application-level option to disable backend connection reuse for generated routes is one contained solution. That could use an application-scoped serversTransport, although it may fit better within the endpoint-mode work already planned in #3480?

This is different from #3480 and #4421, which concern Swarm VIPs that do not route. Here the VIP routes correctly; the concern is reuse of an existing backend connection. Moving to endpoint-mode: dnsrr would not affect a connection already in the pool.

Provide environment information
Operating System:
- Linux Docker Swarm
- Both x86_64 and arm64 remote servers
- Fedora 43 and Ubuntu 24.04 hosts

Dokploy version:
- v0.30.4

Docker:
- Docker Engine 29.7.2 on most nodes
- Docker Engine 29.2.1 on one manager

Hosting:
- Self-hosted multi-node Swarm

Application/service example here:
- Crawl4AI HTTP API
- Docker-image deployment with replicas
- Dokploy-managed Traefik domain routing to the Swarm service VIP
- Container health check enabled
- Start-first rolling updates
Which area(s) are affected? (Select all that apply)

Docker, Traefik, Remote server, Local Development, Cloud Version

Are you deploying the applications where Dokploy is installed or on a remote server?

Both

Additional context

No response

Will you send a PR to fix it?

Yes

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with createServiceConfig in packages/server/src/utils/traefik/application.ts at the linked lines, then reproduce the rolling-update scenario with multiple replicas and keep-alive traffic. Compare the generated Traefik route with the application-scoped serversTransport idea and review the related endpoint-mode work in #3480 and #4421. Done means requests no longer reach a task after Swarm removes it from membership through a reused backend connection.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, typescript
Domain
backend, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.