Consider setting TCP_QUICKACK instead of TCP_NODELAY when possible

TCP_QUICKACK disables delayed acknowledgements (one of the more problematic parts of the implementation of Nagle's algorithm) but keeps small packet buffering, whereas TCP_NODELAY disables both (reducing throughput on small writes).

As put by Nagle himself:

Turning on TCP_NODELAY has similar effects, but can make throughput worse for small writes. If you write a loop which sends just a few bytes (worst case, one byte) to a socket with "write()", and the Nagle algorithm is disabled with TCP_NODELAY, each write becomes one IP packet. This increases traffic by a factor of 40, with IP and TCP headers for each payload. Tinygram prevention won't let you send a second packet if you have one in flight, unless you have enough data to fill the maximum sized packet. It accumulates bytes for one round trip time, then sends everything in the queue. That's almost always what you want. If you have TCP_NODELAY set, you need to be much more aware of buffering and flushing issues.

None of this matters for bulk one-way transfers, which is most HTTP today. (I've never looked at the impact of this on the SSL handshake, where it might matter.)

Short version: set TCP_QUICKACK. If you find a case where that makes things worse, let me know.

Unfortunately, when you search online you may find that TCP_QUICKACK is Linux-only. This is not true! Doing a setsockopt with option 12 (the same define as on Linux) under Windows works (although if it does something I don't know), at least on Windows 10. Additionally, it's seemingly not needed on OS X at all ("on my OSX MacBook Air however the RPC call needed only 3ms!").

However a second issue arises: TCP_QUICKACK can turn itself off. The solution to this is seemingly turning it back on after every recv call.

See also: https://github.com/urllib3/urllib3/issues/746, and this RFC.

No files or tests are named. Start by locating Trio's TCP socket setup and recv paths, then inspect how setsockopt handles TCP_NODELAY and TCP_QUICKACK across Linux, Windows, and macOS. Done means a justified cross-platform change that accounts for QUICKACK disabling itself after recv calls and includes coverage for the supported behavior.