Batch predict using custom models with mandatory parameter fields is not supported in vertex AI

We have custom models that requires passing mandatory parameter field for prediction.
Now this works for online predictions via endpoint in POST request,
but for batch predictions we have to create jsonl file .
The format of each line in jsonl is something like this

```
{"id1":1,"id2":1,"timestamp":"2023-09-28 09:55:47Z","key1":{some dict},"key2":{some dict}}
{...}
```
etc

This throws below error as parameter field is not present.

We tried adding it too but looks it doesn't work either. Can someone let us know to make it work as I see this as a bug.

#### Environment details

- OS type and version:
- Python version: `Python 3.9.1`
- pip version: `23.2.1`
- `google-cloud-aiplatform` version: `1.33.1`

#### Steps to reproduce

1. deploy a custom model with mandatory parameter in the request body for prediction
2. create batch input jsonl file in gcs bucket , use this file as input to batch predict method of the model.
3. Notice the job completion output error in the gcs bucket.

#### Code example

```python
job = model.batch_predict(machine_type="n1-standard-16", max_replica_count=3, job_display_name="test_score", gcs_source="gs://test/batch_predict_requests.jsonl", batch_size=1000, model_parameters=body["parameters"], gcs_destination_prefix="gs://test/output_dir", sync=True)
```

#### Stack trace
```
('Post request fails. Cannot get predictions. Error: Exceeded retries: Non-OK result 422 ({"detail":[{"loc":["body","parameters"],"msg":"field required","type":"value_error.missing"}]}) from server, retry=3, ellapsed=0.13s.', 1000)
```

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

Start at the model.batch_predict entry point and trace how the gcs_source JSONL input and model_parameters are converted into batch prediction requests. Reproduce the job with the supplied custom model parameters and confirm that mandatory parameters reach the prediction request without the server's missing-field error.

Written by the indexing model from the issue text.