[5.1] Flaky Test: TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner

We're seeing intermittent failures in the `TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner` test:
```
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:41395 Is Server Socket Bound: True Testing connectivity to the endpoint created for the server.
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:41395 Endpoint test successful.
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:41395 Listener Thread Started
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:41395 Connection closed but NOT removed (not found): 127.0.0.1:53526
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:41395 New connection accepted: 127.0.0.1:53526 Total connections: 1
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:40853 Is Server Socket Bound: True Testing connectivity to the endpoint created for the server.
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:40853 Endpoint test successful.
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:40853 Listener Thread Started
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:40853 Connection closed but NOT removed (not found): 127.0.0.1:45696
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:40853 New connection accepted: 127.0.0.1:45696 Total connections: 1
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:40853 New connection accepted: 127.0.0.1:45702 Total connections: 2
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:40853 Connection closed and removed: 127.0.0.1:45702
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:41395 New connection accepted: 127.0.0.1:53538 Total connections: 2
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:41395 New connection accepted: 127.0.0.1:53540 Total connections: 3
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:41395 Connection closed and removed: 127.0.0.1:53540
TDSServerEndPoint TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner 0.0.0.0:41395 Connection closed and removed: 127.0.0.1:53538
[xUnit.net 00:00:04.44] Microsoft.Data.SqlClient.Tests.SqlConnectionBasicTests.TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner [FAIL]
[xUnit.net 00:00:04.44] Assert.Equal() Failure: Values differ
[xUnit.net 00:00:04.44] Expected: 1
[xUnit.net 00:00:04.44] Actual: 2
[xUnit.net 00:00:04.44] Stack Trace:
[xUnit.net 00:00:04.45] /_/src/Microsoft.Data.SqlClient/tests/FunctionalTests/SqlConnectionBasicTests.cs(286,0): at Microsoft.Data.SqlClient.Tests.SqlConnectionBasicTests.TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner()
```
I recently added the enhanced TDSServerEndPoint logging that shows the client connections coming and going. Hopefully that helps track down the issue.

In the above logs, we see 2 listener sockets accepting clients (ports 41395 and 40853). I'm not sure if we're expecting 2 listeners.

Here is an example CI run:

https://sqlclientdrivers.visualstudio.com/public/_build/results?buildId=133820&view=logs&j=6dc9fece-c22c-5b30-f35a-8f587584b6fc&t=8f3e5450-4b8e-572e-2076-19a4ce40a073

The failures seem to only occur on Linux.