Low maximum limit for batch jobSpec pushParallelism

On batch jobs processing lots of segments for a table, they often run into Zookeeper conflicts when updating idealState. This causes contention on updates, slowing down everything. To resolve that the pushParallelism on a job spec had to be reduced to ~5, so that it would conflicts less recently. There was another issue y'all resolved which helped ensure progress happened on conflicts.

Being able to upload segments faster will drastically reduce the burden for operating Pinot (backfilling, large segment uploads, or for adjusting existing data).

Start by tracing the job spec pushParallelism limit and the batch-job updates to idealState, then examine how Zookeeper conflicts are handled. Reproduce the contention with a high pushParallelism value if a suitable batch-job path is available. Done means segment uploads can use a higher parallelism without recurring conflicts or loss of progress.

Written by the indexing model from the issue text.