aws / aws/sagemaker-python-sdk

Configurable (or just much longer?) health-check timeout in local mode

未关闭
#3,362 2 条评论 2 个 reaction 已指派 1 人 已被 @nargokul 认领 在 GitHub 查看
component: local mode type: feature request
主要语言
Python
星标
2.3k
派生
1.3k
平均合并
1 天 22 小时
30 天内合并 PR
35

描述

**Describe the feature you'd like**

Today, local-mode endpoint deployment uses a [hard-coded health check time-out](https://github.com/aws/sagemaker-python-sdk/blob/ace07d72f4f44c43fe95b05574968decc7e806ac/src/sagemaker/local/entities.py#L44) of 120s for the container to become healthy.

This does not appear to be consistent with the start-up requirements for actual SageMaker endpoints, and even if it was, it may not be appropriate to assume local environments have similar network bandwidth or compute capabilities to target instance types.

**How would this feature be used? Please describe.**

I'm currently testing a use case with large (e.g. ~5GB+) model archives, and finding local mode deployment fails due to this healthcheck time-out, even though actual SageMaker endpoint deployments succeed without any issue.

If the default timeout was significantly longer, I think it should work okay. If the default timeout was configurable somehow, I could force it to wait longer for my use case.

**Describe alternatives you've considered**

Possible options could include:
- Extending the timeout
- Making the timeout configurable
- Somehow excluding tarball download/extract time from the coverage of the timeout check
- Supporting decompressed local folders as `model_data` targets for local models/endpoints - instead of requiring S3/tarball.

**Additional context**

N/A

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。