Segments downloaded on the server side can be incomplete files

```
// First attempt via segment upload (FAILED due to
2022/06/30 01:17:37.045 INFO [BaseTableDataManager] Download segment: xxx of table: sales_seat_metrics_additive_OFFLINE as crc changes from: 822932084 to: 976745101
2022/06/30 01:17:37.082 INFO [BaseTableDataManager Downloaded tarred segment: xxx for table: yyy from: https://... file length: 21332

// Segment load failed due to EOF Exception.
2022/06/30 15:01:47.857 ERROR [yyy-SegmentRefreshMessageHandler] onError: INTERNAL,
ERROR
java.io.EOFException: null
at org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream.read(GzipCompressorInputStream.java:306) ~[commons-compress-1.21.jar:1.21]
at org.apache.commons.compress.archivers.tar.TarArchiveInputStream.read(TarArchiveInputStream.java:738) ~[commons-compress-1.21.jar:1.21]
at java.io.InputStream.read(InputStream.java:205) ~[?:?]
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1309) ~[commons-io-2.11.0.jar:2.11.0]
at org.apache.commons.io.IOUtils.copy(IOUtils.java:978) ~[commons-io-2.11.0.jar:2.11.0]
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1282) ~[commons-io-2.11.0.jar:2.11.0]
at org.apache.commons.io.IOUtils.copy(IOUtils.java:953) ~[commons-io-2.11.0.jar:2.11.0]
at org.apache.pinot.common.utils.TarGzCompressionUtils.untar(TarGzCompressionUtils.java:167) ~[pinot-common-0.11.0-dev-575.jar:0.11.0-dev-575-ae9a1dc26eaff3be719f9804e23a5

// Second attempt via segment refresh (SUCCEEDED)
2022/06/30 15:13:51.499 INFO [BaseTableDataManager] Download segment:xxx of table: sales_seat_metrics_additive_OFFLINE as crc changes from: 822932084 to: 976745101
2022/06/30 15:13:51.515 INFO [BaseTableDataManager] Downloaded tarred segment: xxx for table: yyy from: https://... file length: 21440
```
In both cases, the server was attempting to refresh segments with CRC from `822932084 to 976745101`; however, the downloaded file lengths for 2 cases are different. (`21332` in the first attempt and `21440` in the second attempt). The first attempt failed due to EOFException and 2nd attempt went through.

We need to work on the following things:

1. We need to investigate why segment downloads can end up having wrong bytes.
2. Add CRC check & re-try for HTTP segment fetcher

Start with the segment download path in BaseTableDataManager and the failure stack through TarGzCompressionUtils. Compare the logged CRC and file-length differences with the HTTP segment fetcher behavior. Done means incomplete segment downloads are detected by CRC and retried, with the EOF failure scenario covered.

Written by the indexing model from the issue text.