Docker swarm with mixed node doesnt remove assigned Ports on Windows Nodes

Description

Hello,

Recently I have been trying out Docker Swarm in a mixed node environment, since I have some Windows containers that I need to run. I then did deploy a stack with some containers on Linux and some on Windows. But you can only connect through the Windows nodes on the first deployment. If you remove the deployment and redeploy it, you can only access the containers through the Linux nodes. Digging into the windows logs it looks like, there is never a request to Windows to delete all ELB's. Thus, when the stack is deployed a second time, it can not use the ports and access is denied on Windows nodes.

Response from Microsoft Support:

Reproduce

1. Create a swarm with one Linux and one Windows node
2. Deploy the example compose file as a stack (example file below)
3. Check if you can access the containers at http://<node_ip>/ & http://<node_ip>/nginx from each of the nodes. Both links from both nodes should bring up a website.
4. Remove the stack.
5. Deploy the stack again.
6. Check if you can access the containers at http://<node_ip>/ & http://<node_ip>/nginx from each of the nodes. Now you can only access the sites from the Linux node.

services:
  traefik:
    image: "traefik:v3.1"
    command:
      - "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--providers.swarm.exposedByDefault=false"
      - "--entryPoints.web.address=:80"
      - "--providers.swarm.endpoint=unix:///var/run/docker.sock"
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.role == manager

  nginx:
    image: nginx
    deploy:
      mode: replicated
      replicas: 1
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.nginx.rule=PathPrefix(`/nginx`)"
        - "traefik.http.routers.nginx.entryPoints=web"
        - "traefik.http.routers.nginx.service=nginx-service"
        - "traefik.http.routers.nginx.middlewares=strip_nginx"
        - "traefik.http.middlewares.strip_nginx.stripprefix.prefixes=/nginx"
        - "traefik.http.services.nginx-service.loadbalancer.server.port=80"

  aspnet:
    image: mcr.microsoft.com/dotnet/framework/samples:aspnetapp
    deploy:
      mode: replicated
      replicas: 1
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.aspnet.rule=PathPrefix(`/`)"
        - "traefik.http.routers.aspnet.entryPoints=web"
        - "traefik.http.routers.aspnet.service=aspnet-service"
        - "traefik.http.services.aspnet-service.loadbalancer.server.port=80"

Expected behavior

No matter how many times I redeploy the stack, it should always be available from all nodes.

docker version

Windows node:

Client:
 Version:           27.1.1
 API version:       1.46
 Go version:        go1.21.12
 Git commit:        6312585
 Built:             Tue Jul 23 19:57:57 2024
 OS/Arch:           windows/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.1.1
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       cc13f95
  Built:            Tue Jul 23 19:56:46 2024
  OS/Arch:          windows/amd64
  Experimental:     false

Linux node:

Client: Docker Engine - Community
 Version:           27.5.1
 API version:       1.47
 Go version:        go1.22.11
 Git commit:        9f9e405
 Built:             Wed Jan 22 13:41:17 2025
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.5.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.11
  Git commit:       4c9b3b0
  Built:            Wed Jan 22 13:41:17 2025
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.25
  GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc:
  Version:          1.2.4
  GitCommit:        v1.2.4-0-g6c52b3f
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Windows node:

Client:
 Version:    27.1.1
 Context:    default
 Debug Mode: false

Server:
 Containers: 5
  Running: 4
  Paused: 0
  Stopped: 1
 Images: 4
 Server Version: 27.1.1
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local splunk syslog
 Swarm: active
  NodeID: muysyf45fu319eb0dg92vbhil
  Is Manager: false
  Node Address: 193.246.152.8
  Manager Addresses:
   193.246.152.34:2377
   193.246.152.35:2377
   193.246.152.41:2377
 Default Isolation: process
 Kernel Version: 10.0 20348 (20348.1.amd64fre.fe_release.210507-1500)
 Operating System: Microsoft Windows Server Version 21H2 (OS Build 20348.3091)
 OSType: windows
 Architecture: x86_64
 CPUs: 4
 Total Memory: 16GiB
 Name: vServer008
 ID: d396a495-8baa-4d9b-811e-f0c0807d3993
 Docker Root Dir: C:\ProgramData\docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Linux node:

Client: Docker Engine - Community
 Version:    27.5.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.20.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.4
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 11
  Running: 7
  Paused: 0
  Stopped: 4
 Images: 7
 Server Version: 27.5.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: active
  NodeID: fpq53gsjyhqa936ovq774v8iw
  Is Manager: true
  ClusterID: osh0r36dqs9e05i7xt8vpag1w
  Managers: 3
  Nodes: 5
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 193.246.152.34
  Manager Addresses:
   193.246.152.34:2377
   193.246.152.35:2377
   193.246.152.41:2377
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.2.4-0-g6c52b3f
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.1.0-31-amd64
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 31.34GiB
 Name: vServer034
 ID: cacd8662-8452-424a-88d0-fcf436faeabb
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

Reproduce the mixed Linux/Windows Docker Swarm setup with the supplied stack, then remove and redeploy it while checking Windows node logs for the missing ELB deletion request. Compare port availability and access from both nodes across deployments; done means repeated redeployments keep both services reachable through every node.