[Segment ingestion job] Multi-threaded segment copy from staging dir to final destination

SparkIngestionJob uses single threaded copy for segments copy from staging dir to final deep store dir(e.g. S3):
https://github.com/apache/pinot/blob/7ce8b756dff2b153151517afd552759e44d46c75/pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-spark/src/main/java/org/apache/pinot/plugin/ingestion/batch/spark/SparkSegmentGenerationJobRunner.java#L345

For a few hundred segments being uploaded, it can take forever. In this case, we should consider generating a list of source and destination paths for segment copy then use RDD to parallel this job.

Start in pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-spark/src/main/java/org/apache/pinot/plugin/ingestion/batch/spark/SparkSegmentGenerationJobRunner.java around line 345, where segments are copied from the staging directory to the final deep-store directory. Review how source and destination paths are formed, then evaluate the proposed RDD-based parallel copy. Done means segment uploads are parallelized for large batches without changing their destinations.

Written by the indexing model from the issue text.