Can we reliably detect an incomplete first commit vs index corruption? [LUCENE-5024]

Normally, if something bad happens (OS, JVM, hardware crashes) while
IndexWriter is committing, we will just fallback to the prior commit
and no intervention necessary from the app.

But if that commit is the first commit, then on restart IndexWriter
will now throw CorruptIndexException, as of #5803.

Prior to #5803, in #3886, we used to try to detect the
corrupt first commit, but that logic was dangerous and could result in
falsely believing no index is present when one is, e.g. when transient
IOExceptions are thrown due to file descriptor exhaustion.

But now two users have hit this change ... see "CorruptIndexException
when opening Index during first commit" and "Calling
IndexWriter.commit() immediately after creating the writer", both on
java-user.

It would be nice to get back to not marking an incomplete first commit
as corruption ... but we have to proceed carefully.

---
Migrated from [LUCENE-5024](https://issues.apache.org/jira/browse/LUCENE-5024) by Michael McCandless (@mikemccand), updated May 09 2016

Start with IndexWriter's first-commit and restart handling, then trace the CorruptIndexException path described in the issue. Compare the incomplete-first-commit case with genuine index corruption and determine what tests cover both; done means reliably distinguishing them without treating transient IOExceptions as an empty index.

Written by the indexing model from the issue text.