Error while testing the llama2-7b model deployment in Vertex AI

Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

- Search the issues already opened: https://github.com/googleapis/python-aiplatform/issues
- Search StackOverflow: https://stackoverflow.com/questions/tagged/google-cloud-platform+python

If you are still having issues, please be sure to include as much information as possible:

#### Environment details

- OS type and version: Ubuntu 22.04
- Python version: 3.10.12
- pip version: pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
- `google-cloud-aiplatform` version:

```Name: google-cloud-aiplatform
Version: 1.44.0
Summary: Vertex AI API client library
Home-page: https://github.com/googleapis/python-aiplatform
Author: Google LLC
Author-email: googleapis-packages@google.com
License: Apache 2.0
Location: /home/kishor/.local/lib/python3.10/site-packages
Requires: google-api-core, google-auth, google-cloud-bigquery, google-cloud-resource-manager, google-cloud-storage, packaging, proto-plus, protobuf, shapely
Required-by:
```

#### Steps to reproduce

1. Deploy llama2-7B-chat-001 through Vertex AI Model Garden
2. Once the deployment is completed, follow the **Sample Request** Python example.

#### Code example

```
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# [START aiplatform_predict_custom_trained_model_sample]
from typing import Dict, List, Union

from google.cloud import aiplatform
from google.protobuf import json_format
from google.protobuf.struct_pb2 import Value

def predict_custom_trained_model_sample(
project: str,
endpoint_id: str,
instances: Union[Dict, List[Dict]],
location: str = "us-central1",
api_endpoint: str = "us-central1-aiplatform.googleapis.com",
):
"""
`instances` can be either single instance of type dict or a list
of instances.
"""
# The AI Platform services require regional API endpoints.
client_options = {"api_endpoint": api_endpoint}
# Initialize client that will be used to create and send requests.
# This client only needs to be created once, and can be reused for multiple requests.
client = aiplatform.gapic.PredictionServiceClient(client_options=client_options)
# The format of each instance should conform to the deployed model's prediction input schema.
instances = instances if isinstance(instances, list) else [instances]
instances = [
json_format.ParseDict(instance_dict, Value()) for instance_dict in instances
]
parameters_dict = {}
parameters = json_format.ParseDict(parameters_dict, Value())
endpoint = client.endpoint_path(
project=project, location=location, endpoint=endpoint_id
)

response = client.predict(
endpoint=endpoint, instances=instances, parameters=parameters
)

print(" deployed_model_id:", response.deployed_model_id)
# The predictions are a google.protobuf.Value representation of the model's predictions.
predictions = response.predictions
for prediction in predictions:
print(" prediction:", dict(prediction))

# [END aiplatform_predict_custom_trained_model_sample]

predict_custom_trained_model_sample(
project="xxxxxxxxx",
endpoint_id="xxxxxxxxx",
location="us-xxxxx",
instances=[
{
"prompt": "What is the sum of 999 and 1?",
"max_tokens" : 1000,
"temperature" : 0
}
]
)
```

#### Stack trace
```
Traceback (most recent call last):
File "/home/kishor/.local/lib/python3.10/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
return callable_(*args, **kwargs)
File "/home/kishor/.local/lib/python3.10/site-packages/grpc/_channel.py", line 1176, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/kishor/.local/lib/python3.10/site-packages/grpc/_channel.py", line 1005, in _end_unary_response_blocking
raise _InactiveRpcError(state) # pytype: disable=not-instantiable
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.NOT_FOUND
details = "Endpoint `projects/xxxxxxxxx/locations/us-xxxxx/endpoints/xxxxxxxxx` not found."
debug_error_string = "UNKNOWN:Error received from peer ipv4:142.250.76.42:443 {grpc_message:"Endpoint `projects/xxxxxxxxx/locations/us-xxxxx/endpoints/xxxxxxxxx` not found.", grpc_status:5, created_time:"2024-03-18T13:01:09.681075746+05:30"}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/kishor/Interplai_GPT/GCP/prompt_002.py", line 74, in
predict_custom_trained_model_sample(
File "/home/kishor/Interplai_GPT/GCP/prompt_002.py", line 60, in predict_custom_trained_model_sample
response = client.predict(
File "/home/kishor/.local/lib/python3.10/site-packages/google/cloud/aiplatform_v1/services/prediction_service/client.py", line 836, in predict
response = rpc(
File "/home/kishor/.local/lib/python3.10/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
return wrapped_func(*args, **kwargs)
File "/home/kishor/.local/lib/python3.10/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.NotFound: 404 Endpoint `projects/xxxxxxxxx/locations/us-xxxxx/endpoints/xxxxxxxxx` not found.
```

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!