aws / aws/amazon-sagemaker-examples
Received server error (500) from primary and could not load the entire response body from endpoint
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
[ERROR] ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (500) from primary and could not load the entire response body. See https://eu-west-3.console.aws.amazon.com/cloudwatch/home?region=eu-west-3#logEventViewer:group=/aws/sagemaker/Endpoints/pytorch-inference-2023-07-10-09-51-02-299 in account 086892845792 for more information. Traceback (most recent call last): File "/var/task/lambda_function.py", line 442, in lambda_handler pred_prob = invoke_endpoint_with_idx(endpointname = ENDPOINT_NAME, target_id = transaction_id, subgraph_dict = subgraph_dict, n_feats = transaction_embed_value_dict) File "/var/task/lambda_function.py", line 314, in invoke_endpoint_with_idx response = runtime.invoke_endpoint(EndpointName=endpointname, File "/opt/python/lib/python3.8/site-packages/botocore/client.py", line 508, in _api_call return self._make_api_call(operation_name, kwargs) File "/opt/python/lib/python3.8/site-packages/botocore/client.py", line 911, in _make_api_call raise error_class(parsed_response, operation_name) [enter image description here](https://i.stack.imgur.com/S5H8f.png) please i got this error while running the following code https://github.com/awslabs/realtime-fraud-detection-with-gnn-on-dgl/tree/main/src/sagemaker. once shouted the endpoint and called it from lambda I received that, I redid several times it's always the same error, can you help me please
I run the preprocessing code then train the model on sagemaker , after that i deploy it ( the code is from aws : https://github.com/awslabs/realtime-fraud-detection-with-gnn-on-dgl/tree/main/src/sagemaker) i invoke the endpoint with lambda and i receive this server error, in the first it's was ok then it's became like this , I tried everything but still the same problem,

Contributor guide
Research direction
Start with the SageMaker example under src/sagemaker and inspect lambda_function.py at the invocation path around lines 314 and 442. Check the linked CloudWatch endpoint logs alongside the Lambda request and response, then compare them with the example's deployment and invocation flow. Done means the deployed endpoint can be invoked from Lambda without the 500 response and the full response body is received.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, pytorch
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100