googleapis / googleapis/python-aiplatform

Endpoint prediction GRPC error

Abierto
#1,849 5 comentarios 0 reacciones 1 asignado Reclamado por @kthytang Ver en GitHub
api: vertex-ai
Lenguaje dominante
Python
Estrellas
905
Forks
465
Merge medio
1 d 13 h
PR fusionados (30 d)
44

Descripción

Getting a hard to debug error when attempting to send predictions to deployed Endpoint. The endpoint is serving predictions just fine via `curl` but fails when using the client library. Other methods on the endpoint work fine, just not prediction it seems.

#### Environment details

- OS type and version: Ubuntu 20.04.5 LTS
- Python version: 3.9.7
- pip version: 22.3
- `google-cloud-aiplatform` version: 1.19.1

#### Steps to reproduce

1. Create Vertex Endpoint for custom trained model
2. Send prediction request

#### Code example

```python
aiplatform.init(project="my-project", location="my-location")
endpoint = aiplatform.Endpoint("endpoint-id")
predictions = endpoint.predict(instances=[{...}, {...}, {...}], parameters={})
print(predictions)
```

#### Stack trace
```
Traceback (most recent call last):
File "/home/jupyter/.pyenv/versions/***/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
return callable_(*args, **kwargs)
File "/home/jupyter/.pyenv/versions/***/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/jupyter/.pyenv/versions/***/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
details = "Received RST_STREAM with error code 2"
debug_error_string = "UNKNOWN:Error received from peer ipv4:***:443 {grpc_message:"Received RST_STREAM with error code 2", grpc_status:13, created_time:"2022-12-13T18:05:21.914025462+00:00"}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/jupyter/***/***-model/predict.py", line 12, in
predictions = endpoint.predict(instances=input_dict["instances"], parameters=input_dict["parameters"])
File "/home/jupyter/.pyenv/versions/***/lib/python3.9/site-packages/google/cloud/aiplatform/models.py", line 1503, in predict
prediction_response = self._prediction_client.predict(
File "/home/jupyter/.pyenv/versions/***/lib/python3.9/site-packages/google/cloud/aiplatform_v1/services/prediction_service/client.py", line 589, in predict
response = rpc(
File "/home/jupyter/.pyenv/versions/***/lib/python3.9/site-packages/google/api_core/gapic_v1/method.py", line 154, in __call__
return wrapped_func(*args, **kwargs)
File "/home/jupyter/.pyenv/versions/***/lib/python3.9/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.InternalServerError: 500 Received RST_STREAM with error code 2
```

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.