Supporting "Windows Subsystem for Linux" in Trio

Windows now has this "WSL" thing, that lets you (allegedly) run unmodified Linux apps on Windows. Since it's a from-scratch reimplementation of the Linux APIs, it doesn't actually work exactly like Linux; it's effectively a new platform.

So far supporting WSL hasn't been a high priority, and we don't currently test on it or anything. I don't know how high a priority it should be. But all else being equal, it certainly would be nicer to support it than not. And @dd-dent and @codypiersall were experimenting with it today in gitter, and I guess more people will probably experiment in the future, so let's have an issue to track the status :-).

The first discovery was that non-blocking socketpair() sockets are totally busted on Windows 1803: https://github.com/Microsoft/WSL/issues/3100

Trio currently uses non-blocking socketpair() sockets internally, and they're used extensively in the test suite, so that's kind of a non-starter right now. Fortunately this is apparently fixed in Windows 1809.

Cody upgraded to 1809 and tried again, reporting:

The test suite made it much further after upgrading Windows. Still some failures: trio/tests/test_highlevel_open_tcp_listeners.py::test_open_tcp_listeners_rebind FAILED but IIRC SO_REUSEADDR behavior differs on Windows from Unix, and maybe WSL confroms to the Windows behavior instead of Linux (which is different from BSD too, I think??? Maybe not...)

The bigger issue is that the test trio/tests/test_highlevel_socket.py::test_SocketStream_send_all hangs. It's hanging at the recevier() line await wait_all_tasks_blocked() and the sender is stuck at trio/_highlevel_socket.py, the first call to self.socket.send(remaining) in the send_all() loop. I guess the issue is that await wait_all_tasks_blocked is not working for some reason, but I haven't the slightest clue why. Seems like that wouldn't be platform-specific. I also could definitely be misdiagnosing the problem.

So... promising progress, but still some issues that haven't been fully analyzed.

If WSL was going to become an "officially supported" platform for trio, then eventually we'll need to figure out how to run CI on it. This issue has some info on a possible way to run WSL on azure pipelines for testing: https://github.com/Microsoft/azure-pipelines-image-generation/issues/478. I haven't tried it, no idea if it works, and currently the newest windows version available on azure pipelines is 1803, so probably we would need to wait for 1809 to be rolled out anyway. (Plus we know trio doesn't work on WSL right now anyway, so there's not much point in worrying about CI until that's fixed.)

Start by running the Trio test suite under WSL, focusing on trio/tests/test_highlevel_open_tcp_listeners.py::test_open_tcp_listeners_rebind and trio/tests/test_highlevel_socket.py::test_SocketStream_send_all. Read trio/_highlevel_socket.py around the send_all() loop and investigate the socketpair and wait_all_tasks_blocked behavior described in the issue. Done means the failures are understood and WSL support requirements are documented or the affected tests pass.