AWS Sagemaker batchtransform job fails if the test dataset size is greater than or equal to 5 MB

Hi Team,

Created a custom algorithm by following the sample notebook bring_your_own_container_as_algorithm from the below link
[https://github.com/aws/amazon-sagemaker-examples/blob/master/aws_marketplace/creating_marketplace_products/Bring_Your_Own-Creating_Algorithm_and_Model_Package.ipynb](url)

Batch-transform job fails if the test dataset size is greater-than or equal to 5 MB, for this i played with additional configuration parameters which are available while creating batch-transform job(max_payload,max_concurrent_transforms,split_type=line) and other params, but didn't work, then i changed nginx.conf parametes client_max_body_size 0;client_body_buffer_size 100, subrequest_output_buffer_size 100m; keepalive_timeout 3600; proxy_read_timeout 12000s; i tried different combinations of these didn't work, after that i came across split the dataset into chuncks form [https://aws.amazon.com/blogs/machine-learning/aws-internal-use-case-evaluating-and-adopting-amazon-sagemaker-within-aws-marketing/](url) page, i tried it on predictor.py file
if flask.request.content_type == 'text/csv':
data = flask.request.data
s = io.StringIO(data.decode('utf-8'))
print("started new process")
data = pd.read_csv(s)
df_size=data.memory_usage(deep=True).sum()
chuncks=math.ceil(math.ceil(df_size/1024/1024)/5)
df_split = np.array_split(data, chuncks)
after trying all the above solutions, batchtransform job fails with "too much data for max payload size" or "413 Request Entity Too Large " or 502 badgateway or timeout error

Start with Bring_Your_Own-Creating_Algorithm_and_Model_Package.ipynb and predictor.py, then reproduce a SageMaker batch-transform job using a test dataset at least 5 MB. Compare the reported payload, 413, 502, and timeout failures with the batch-transform and container configuration; done means the job completes successfully for that dataset size.

Written by the indexing model from the issue text.