A Sharding Split Policy For Load-Sharing Index Writes [LUCENE-2435]

(Please note that the feature described in #3499 is required reading for this issue.)

The sharding split policy builds on the remoting split policy, treating each sub-index as a shard of the split index. Optionally, the super-index on which the split writer is built may also be considered to be a shard. The motivation for this policy is to allow a given index to be partitioned across multiple nodes (that define a cluster), which helps to (a) load-balance document writes across the nodes in that cluster and (b) serve search requests from a (narrower) set of sub-indices (that the split rule routes the query to).

Unlike other sharding solutions for Lucene, this approach gives users the ability to (a) pick and choose the types of directories underlying the mirror, and (b) add/remove shards on the fly.

---
Migrated from [LUCENE-2435](https://issues.apache.org/jira/browse/LUCENE-2435) by Karthick Sankarachary
Attachments: [LUCENE-2435.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-2435/LUCENE-2435.patch)
Linked issues:
- #3499

Read linked issue #3499 first, since this issue states that its feature is required background. Then review the attached LUCENE-2435.patch and compare its proposed sharding split policy with the remoting split policy described here. Done means defining and implementing the load-sharing sharding behavior, including shard selection and adding or removing shards on the fly.

Written by the indexing model from the issue text.