After disconnecting and reconnecting a container, the exposed port does not rebind to the new host port

Description
After disconnecting and reconnecting a container to the bridge network with a dynamic host port binding, the newly assigned host port is not reachable and the container is still reachable via the old port binding.

Steps to reproduce the issue:

1. Run docker run --name mynginx1 -p 80 -d nginx
2. Inspect the container and telnet the binded port
3. Run docker network disconnect bridge <container Id>
4. Run docker network connect bridge <container Id>

Describe the results you received:

a. Try to telnet container on newly binded host port (Identify it is unreachable)
b. Try to telnet container on old binded host port (Identify it is reachable)

Describe the results you expected:
a. Try to telnet container on newly binded host port (Identify it is reachable)
b. Try to telnet container on old binded host port (Identify it is unreachable)

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client: Docker Engine - Community
 Cloud integration  0.1.18
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:00:27 2020
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 10
  Running: 4
  Paused: 0
  Stopped: 6
 Images: 1755
 Server Version: 19.03.13
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.104-microsoft-standard
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 24.95GiB
 Name: docker-desktop
 ID: TAE7:3G7R:YSMF:AJDH:OPMK:CFTK:74OY:LJ3M:OFOH:KI4S:TL4I:CXO7
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 74
  Goroutines: 71
  System Time: 2021-01-04T20:38:06.7034025Z
  EventsListeners: 2
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

Additional environment details (AWS, VirtualBox, physical, etc.):
We are having the same behavior across different types of host OS