aws / aws/sagemaker-python-sdk

ModelBuilder.build() fails with ValueError in _tmpdir: model_path directory never created in passthrough path

已关闭
#5,748 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
2.3k
派生
1.3k
平均合并
1 天 22 小时
30 天内合并 PR
35

描述

## Description

`ModelBuilder.build()` fails with a `ValueError` when repacking model artifacts because `model_path` (which defaults to `/tmp/sagemaker/model-builder/`) is assigned to `sagemaker_session.settings._local_download_dir` but is never created on disk in the passthrough build path.

## Steps to Reproduce

```python
from sagemaker.serve.model_builder import ModelBuilder, SourceCode

mb = ModelBuilder(
image_uri=sklearn_image,
s3_model_data_url="s3://bucket/path/model.tar.gz",
role_arn=role,
sagemaker_session=session,
source_code=SourceCode(
source_dir="./inference/",
entry_script="inference.py"
),
)

model = mb.build() # <-- ValueError here
```

## Error

```
ValueError: Inputted directory for storing newly generated temporary directory does not exist:
'/tmp/sagemaker/model-builder/108d3308375c11f19f96222d70a99c6e'
```

## Root Cause

In `model_builder.py`, `model_path` defaults to a UUID-based temp path (line 236):

```python
model_path: Optional[str] = field(
default_factory=lambda: "/tmp/sagemaker/model-builder/" + uuid.uuid1().hex,
)
```

During `_build_single_modelbuilder()`, this path is assigned to the session settings (line 2465):

```python
self.sagemaker_session.settings._local_download_dir = self.model_path
```

The **only** `os.makedirs(self.model_path)` call exists inside `_save_model_inference_spec()` (line 1257-1258), but the passthrough build path (`_build_for_passthrough()`) **never calls** `_save_model_inference_spec()`. So the directory doesn't exist when the code reaches:

```
_build_for_passthrough() → _create_model() → _create_sagemaker_model()
→ _prepare_container_def() → _prepare_container_def_base() → _upload_code()
→ repack_model() → _tmpdir(directory=local_download_dir) → ValueError
```

## Suggested Fix

**Option A (targeted):** In `_build_single_modelbuilder()`, ensure `model_path` exists right after assigning it to `local_download_dir` (after line 2465):

```python
self.sagemaker_session.settings._local_download_dir = self.model_path
if self.model_path and not self.model_path.startswith("s3://"):
os.makedirs(self.model_path, exist_ok=True)
```

**Option B (defensive):** In `_tmpdir()` (`common_utils.py`), create the directory instead of raising:

```python
if directory is not None:
os.makedirs(directory, exist_ok=True)
```

Option A is more targeted. Option B is more defensive and would prevent similar issues from other callers.

## Workaround

```python
import os
os.makedirs(mb.model_path, exist_ok=True)
model = mb.build()
```

## Environment

- SageMaker Python SDK v3 (latest)
- Python 3.12
- Amazon Linux 2023

贡献指南

打开贡献指南

调研方向

从 model_builder.py 中的 _build_single_modelbuilder() 开始,跟踪经过 _build_for_passthrough() 的 passthrough 路径。检查 common_utils.py 中的 _tmpdir() 并复现 ModelBuilder.build() 示例;完成的标志是 passthrough build 在重新打包 artifacts 时不再引发 ValueError。

由索引模型根据 Issue 内容生成。

评估

技术栈
aws, python
领域
cloud, machine-learning
Issue 类型
缺陷
难度
2/5
预计耗时
1-3 小时
活跃度
冷清
描述清晰度
描述清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。