aws / aws/sagemaker-python-sdk

ModelBuilder.build() fails with ValueError in _tmpdir: model_path directory never created in passthrough path

Closed
#5,748 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.3k
Forks
1.3k
Avg merge
2d 2h
Merged PRs (30d)
32

Description

## Description

`ModelBuilder.build()` fails with a `ValueError` when repacking model artifacts because `model_path` (which defaults to `/tmp/sagemaker/model-builder/`) is assigned to `sagemaker_session.settings._local_download_dir` but is never created on disk in the passthrough build path.

## Steps to Reproduce

```python
from sagemaker.serve.model_builder import ModelBuilder, SourceCode

mb = ModelBuilder(
image_uri=sklearn_image,
s3_model_data_url="s3://bucket/path/model.tar.gz",
role_arn=role,
sagemaker_session=session,
source_code=SourceCode(
source_dir="./inference/",
entry_script="inference.py"
),
)

model = mb.build() # <-- ValueError here
```

## Error

```
ValueError: Inputted directory for storing newly generated temporary directory does not exist:
'/tmp/sagemaker/model-builder/108d3308375c11f19f96222d70a99c6e'
```

## Root Cause

In `model_builder.py`, `model_path` defaults to a UUID-based temp path (line 236):

```python
model_path: Optional[str] = field(
default_factory=lambda: "/tmp/sagemaker/model-builder/" + uuid.uuid1().hex,
)
```

During `_build_single_modelbuilder()`, this path is assigned to the session settings (line 2465):

```python
self.sagemaker_session.settings._local_download_dir = self.model_path
```

The **only** `os.makedirs(self.model_path)` call exists inside `_save_model_inference_spec()` (line 1257-1258), but the passthrough build path (`_build_for_passthrough()`) **never calls** `_save_model_inference_spec()`. So the directory doesn't exist when the code reaches:

```
_build_for_passthrough() → _create_model() → _create_sagemaker_model()
→ _prepare_container_def() → _prepare_container_def_base() → _upload_code()
→ repack_model() → _tmpdir(directory=local_download_dir) → ValueError
```

## Suggested Fix

**Option A (targeted):** In `_build_single_modelbuilder()`, ensure `model_path` exists right after assigning it to `local_download_dir` (after line 2465):

```python
self.sagemaker_session.settings._local_download_dir = self.model_path
if self.model_path and not self.model_path.startswith("s3://"):
os.makedirs(self.model_path, exist_ok=True)
```

**Option B (defensive):** In `_tmpdir()` (`common_utils.py`), create the directory instead of raising:

```python
if directory is not None:
os.makedirs(directory, exist_ok=True)
```

Option A is more targeted. Option B is more defensive and would prevent similar issues from other callers.

## Workaround

```python
import os
os.makedirs(mb.model_path, exist_ok=True)
model = mb.build()
```

## Environment

- SageMaker Python SDK v3 (latest)
- Python 3.12
- Amazon Linux 2023

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.