Repeated swarm connects may be causing connectivity issues

#### Version information:
go-ipfs version: 0.4.19-
Repo version: 7
System version: amd64/linux
Golang version: go1.11.5

#### Type:
- "bug": If what you are filing is a bug.

#### Description:
I ran into an issue where repeated swarm connects between two nodes may have caused connectivity issues.

Essentially, I was running the following system service every 1 minute on my host node

```
[Unit]
Description=Job that periodically connects this IPFS node to the gateway node
[Service]
ExecStart=/home/yourUserName/go/bin/ipfs swarm connect /ip4/GGG.GGG.GGG.GGG/tcp/4001/ipfs/gatewayID
Environment="IPFS_PATH=/ipfs"
```

I was then running a complementary version of this service on the gateway node, connecting it to the host node.

Initially on node reboots, things appeared to be working fine, but then after around 30 minutes or so, the network was essentially unable to find new content as it was added to the host node.

Disabling the system service above seems to have fixed my problems, although I filed this bug report at the request of @Stebalien so he could look into a hunch he had.