swarm: potential issue with dial limiter

So this isnt necessarily an implementation bug, but rather a design bug. While investigating connectivity issues i put together a log scraper coalescer thingy that gathers information about each overall dial attempt and each of its individual dials. The full log is here, and should be pretty interesting for anyone who cares about dials.

I'm making this issue because I see a lot of dials like this one:

{
  "dials": [
    {
      "targetAddr": "/ip4/127.0.0.1/tcp/4001",
      "result": "",
      "error": "dial tcp4 127.0.0.1:4001: connect: connection refused",
      "duration": "196.56µs",
      "start": "2018-06-07T02:56:33.091027367-07:00"
    },
    {
      "targetAddr": "/ip6/::1/tcp/4001",
      "result": "",
      "error": "dial tcp6 [::1]:4001: connect: connection refused",
      "duration": "453.875µs",
      "start": "2018-06-07T02:56:33.090807478-07:00"
    },
    {
      "targetAddr": "/ip4/94.24.55.20/tcp/4001",
      "result": "",
      "duration": "3.447658049s",
      "start": "2018-06-07T02:56:33.090940097-07:00"
    }
  ],
  "success": true,
  "duration": "29.289350417s",
  "target": "QmSz8K4YJYwQYvsDjgsYQTHhyf3rg3BUNkpvGqmxq8sDHb",
  "err": "",
  "start": "2018-06-07T02:56:07.249681968-07:00"
}

Where the overall dial takes about 30 seconds, but the sum of the dials involved take only ~3.5s.

Then theres also this one:

{
  "dials": [
    {
      "targetAddr": "/ip6/::1/tcp/4001",
      "result": "",
      "error": "dial tcp6 [::1]:4001: connect: connection refused",
      "duration": "577.624µs",
      "start": "2018-06-07T02:55:36.981039644-07:00"
    },
    {
      "targetAddr": "/ip4/127.0.0.1/tcp/4001",
      "result": "",
      "error": "dial tcp4 127.0.0.1:4001: connect: connection refused",
      "duration": "401.141µs",
      "start": "2018-06-07T02:55:36.981775784-07:00"
    }
  ],
  "success": false,
  "duration": "1m0.001597724s",
  "target": "Qmdx7w4sYJirLNCXpKtyfMsE3ZHzNVSnH94Ej4QFphDHxC",
  "err": "<peer.ID Qb3Xmt> --> <peer.ID dx7w4s> dial attempt failed: dial tcp4 127.0.0.1:4001: connect: connection refused",
  "start": "2018-06-07T02:55:36.074255709-07:00"
}

Where the overall dial operation takes a full minute, but only contains two dials, each of which took ~0 time.

No source file, test, or entry point is named. Start by reviewing the dial-limiter behavior alongside the linked dial logs, then define the expected relationship between the overall dial timeout and its individual dials before proposing tests for the desired behavior.