Optionally cleaning up orphan mosh-servers on connection

Related to #12, #533 and #690.

I'm filing this as an issue separate from #12, even though it has some discussion directly relevant to this , to separate it from the reconnection discussion which is a much larger and quite different issue.

Since my phone likes to die suddenly every now and then, I sometimes end up with lots of orphan mosh-servers. #690 mostly solves this issue for relatively reliable clients, but if the same user has clients that die often(like a mobile phone), but at the same time has highly reliable clients that are sometimes offline for a long time, a single timeout either ends up killing servers for clients that still exist or allows large numbers of orphan servers to accumulate before their cleanup.

Proposed solution

1. A client MAY identify itself to the server with a client id.
   • The client MAY optionally generate one.
   • If the client does not send a client id to the server, that connection MUST NOT be affected by this optional cleanup system.
   • Also, if client id is not supplied, no cleanup of other mosh-servers will be attempted, i.e. that instance of mosh will act as if this system was not implemented at all.
   • Steps 2-3 will take place before starting mosh-server. Step 4 will take place immediately after. Step 5 will happen when SIGUSR2 is received by mosh-server.
   • For backwards compatibility, the client id will be passed as an environment variable, MOSH_CLIENT_ID
2. The client MAY supply along with the client id, its own pid and a list of pids of mosh-clients that client can see, thereby indicating to the server that it knows those mosh-clients to be alive on the system with that same client id. If the list of mosh-clients is not provided, it will interpreted to mean two things:
   • All other mosh-servers listening to this client id should be killed
   • Any future connection with the same client id will kill this connection since it does not supply even its own pid.
   • This mode may be useful for simplified operation on systems that will "never" want to have two simultaneous mosh connections as the same client-side user to the same server-side user on the same server.
3. Mosh will query the mosh connection cache (see step 4) with the following criteria:
   • The client id must match the current connection
   • since the pid of current client is not saved yet, all other client pids match, even if current client's pid was not supplied.
   • If the list of known-alive-mosh-client-pids is not empty, client pid must not match any of those.
   • What remains is a list of those mosh-servers that are:
     • serving clients from the same origin as this one,
     • but whose clients no longer exist, as far as this mosh-client knows.
     • To account for possible timing issues with two closely spaced connections, connections that are less than 30 seconds old will be ignored.
     • Therefore, this client may deduce that those mosh-servers are orphan, and sends a SIGUSR2 to those mosh-servers.
   • Finally, garbage collection is performed: entries corresponding to non-existent mosh-server pids are removed.
4. Mosh-server will save client id to the mosh connection cache, along with the current client's pid if it was supplied, as well as its own (current mosh-server's) pid
   • The default cache would be the filesystem, but others could be implemented.
   • ~/.mosh/servers/mosh_server...
   • file creation time should be used for determining connection creation time for the purposes of excluding newly created sessions from being killed.
5. If a mosh-server receives SIGUSR2:
   • If it has not received a MOSH_CLIENT_ID environment variable, it will ignore the signal.
   • Otherwise, it will remove its own entry in the mosh connection cache, and then terminate.

Start by tracing mosh-client and mosh-server connection startup, the MOSH_CLIENT_ID environment variable, and mosh-server handling of SIGUSR2. Then examine how a filesystem-backed connection cache could record process IDs and creation times; done means implementing the proposed optional cleanup while preserving behavior for clients that do not provide a client ID.