Performance issues when loading data from S3

### Describe the usage question you have. Please include as many useful details as possible.

We are doing one small poc wherein we are comparing the performance when we load parquet files directly from s3 or from local file system

# jupyter notebook code snippet
# jupyter notebook and s3 is in same region

Import pyarrow.dataset as ds
import time

s3Dataset = ds.dataset(‘location’) # s3 or local file system
scanner = s3Dataset.scanner()
batches = scanner.to_batches()
st = time.time()

for batch in batches:
batch.num_rows

et = time.time()

elapsed = et - st

# code ends here

total files 50 and total size of 500 mb

Time taken to read from s3 is 970 sec
Time taken to read from local file system 5 sec

Any insight why S3 is taking so much time?

Is there any settings we are missing when we are reading files from S3 which would give decent performance?

Any help appreciated.

### Component(s)

FlightRPC, Parquet, Python, Other