Make it easier to prevent Server side request forgery (SSRF) attacks

Over the years, we've received many bug bounty reports relating to Server side request forgery (SSRF) attacks. In a nutshell, these attacks use short-lived DNS entries to direct Web hooks and other URLs to internal IP addresses, such as AWS's instance metadata endpoint.

To a large extent, the problem is mitigated by using HTTPS, since a SSL certificate Common Name (CN) must match the hostname. However, there are a number of edge cases where HTTPS doesn't solve the issue. For example:

• DNS rebinding might still enable reconnaissance on the local network, since errors show the difference between "10.1.2.3:4567 unreachable" and "10.1.2.3:4567 reachable but TLS error".
• Some clients or Web hooks may disable SSL certificate verification.

In the past, we've mitigated the problem by:

1. Performing a DNS lookup first for the IP address.
2. If the IP address maps to internal or local networks, reject the request.
3. If the IP address is allowed, make the HTTPS request with the IP address instead of the hostname. To ensure SNI works, we patched net-http to use the original hostname by overriding the hostname= method.

A similar approach is taken by ssrf_filter.

However, with https://github.com/ruby/net-http/pull/36, our net-http patch no longer works because hostname= isn't called when an IP address is used. To handle that, https://github.com/arkadiyt/ssrf_filter/pull/54 introduced an even uglier patch that overrides the Resolv equality methods.

Both hostname= and Resolv patches are a bit ugly, but short of patching the #connect method there's no alternative at the moment.

A better approach might be to invoke some callback in #connect that will allow the caller to resolve the hostname and decide whether the connection should still proceed.

I realize that others might argue that a proxying all external calls via a proxy server is ultimately the right approach, but that's another moving part that requires more setup.

@jeremyevans What do you think about this?

Start by reading net-http's #connect path and the existing hostname= handling, then compare the SSRF mitigation described in the issue with the Resolv workaround in ssrf_filter. The issue suggests a callback for hostname resolution and connection approval, but does not define a settled API or tests; done would require an agreed design and coverage for rejected internal addresses and allowed HTTPS connections.