autogluon / autogluon/autogluon-cloud

Enhance Logging for predict_real_time API in Cloud Predictors

Open
#106 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
42
Forks
17
PR merge metrics
No merged PRs in 30d

Description

#### Description
The `predict_real_time` API currently provides minimal logging information, especially in failure scenarios when using SageMaker endpoints. This lack of detailed logging makes it challenging to diagnose issues or understand the reasons behind failed inference requests. Additionally, the logs available from the cloud endpoint, such as SageMaker, offer limited information beyond HTTP error codes. This situation necessitates improved logging mechanisms to provide users with better visibility and debugging capabilities for their cloud predictors.

#### Expected Behavior
When an inference request fails, detailed error messages or logs should be provided to the user, including but not limited to:
- The specific reason for the failure (e.g., model loading issues, data serialization/deserialization problems, etc.).
- Relevant HTTP error codes along with their descriptions.
- Suggestions or references for troubleshooting common issues.

#### Actual Behavior
Currently, the `predict_real_time` API outputs minimal information in both stdout/stderr and the cloud endpoint logs, mainly limited to HTTP status codes without detailed explanations or context. This minimal feedback loop hinders effective troubleshooting and root cause analysis.

#### Steps to Reproduce
1. Set up a cloud predictor using the AutoGluon-Cloud with a SageMaker endpoint.
2. Attempt to make an inference request using the `predict_real_time` API with a setup that is known to fail (e.g., incorrect input format).
3. Observe the lack of detailed logging information in the event of a failure.

#### Possible Solution
Implement enhanced logging within the `predict_real_time` API to capture and relay detailed error messages and diagnostics information from the underlying cloud service (e.g., SageMaker). This could include:
- Catching exceptions at the API level and enriching them with additional context before re-throwing or logging.
- Enabling configurable log levels for the API, allowing users to opt-in for more verbose logging based on their debugging needs.
- Working closely with cloud service providers to ensure that more detailed error information is made available and propagated through the AutoGluon-Cloud interface.

#### Additional Context
Enhancing the logging detail for cloud predictors not only improves the user experience by providing clear insights into the operational aspects but also significantly reduces the time spent on troubleshooting and support.

Contributor guide

Open the contributing guide

Research direction

Start at the predict_real_time API and reproduce a failed inference request against a SageMaker endpoint using an incorrect input format. Determine what failure context is currently exposed; done should include detailed, actionable error information beyond the HTTP status, including the underlying failure reason where available.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
api, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.