Tests: Meta ticket for testing with external servers

There are several tickets open about tests using external servers/services and the fact that this means that those tests may fail on occation when the service is (temporarily) not available or when something changes in those endpoints.

This meta ticket is meant to bind those tickets together and to discuss the wider pespective of how to solve this.

Related tickets

Related tickets specific to tests with external services:

• https://github.com/WordPress/Requests/issues/315
• https://github.com/WordPress/Requests/issues/419
• https://github.com/WordPress/Requests/issues/496
• https://github.com/WordPress/Requests/issues/639

Related tickets specific to features which can't currently be tested:

• https://github.com/WordPress/Requests/issues/7

Related tickets about documenting how to run the tests (locally with the test servers):

• https://github.com/WordPress/Requests/issues/475
• https://github.com/WordPress/Requests/pull/668

External services providing endpoints

Known external services which provide endpoints for testing status code and other HTTP(S) responses:

• requests-php-tests.herokuapp.com (currently in use in the test suite)
• https://httpbin.org/ (currently in use in the test suite)
• http://portquiz.net/ (currently in use in the test suite)
• https://badssl.com/
• https://mock.codes/
• https://responseco.de/

In addition to this, a local copy of the https://github.com/RequestsPHP/test-server code is being run to provide responses for some tests.

To discuss/investigate

• Are there external services available for all needed endpoints ? I.e. can we abolish our dependency on the Heroku setup / test server repo ?
• Can the tests be made more stable by switching to different services for certain tests ?
• Can the tests be made more stable by having a helper function which will automatically retry a request a limited amount of times if it fails the first time ? (loosely related to feature request #222)
• Can (some of) the tests be made more stable by having a fall-back URL ? I.e. if two different services both provide the response we need for the test, can a second server be tried if the request to the first server failed (via a helper function) ?
• etc

Start by reviewing the related issues #315, #419, #496, #639, #7, and #475, along with pull request #668. Inspect the existing tests using requests-php-tests.herokuapp.com, httpbin.org, portquiz.net, and the local RequestsPHP/test-server copy. Done would require an agreed, stable approach for external-service test dependencies, retries, fallbacks, or replacements.