aws / aws/sagemaker-python-sdk

ModelBuilder fails w/ DJL container using HF_MODEL_ID

Open Beginner friendly
#5,588 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.3k
Forks
1.3k
Avg merge
1d 22h
Merged PRs (30d)
35

Description

**PySDK Version**
- [ ] PySDK V2 (2.x)
- [X] PySDK V3 (3.4.0)

**Describe the bug**
When using ModelBuilder in "passthrough" mode (image_uri + env_vars, no model
or inference_spec), `build()` fails with a ValidationException because a
non-existent S3 path is injected as `ModelDataUrl` in the CreateModel request.

Passthrough mode is the intended path for deploying models on custom/DJL
containers where the container itself handles model loading (e.g. via
HF_MODEL_ID env var pointing to S3).

**Expected behavior**
Expected: Model created with no ModelDataUrl (container loads via HF_MODEL_ID)
Actual: ValidationException -- "Could not find model data at
s3:///model-builder///"

**Suggested Fix**

In `_build_for_passthrough()`, also clear `s3_model_data_url`:

def _build_for_passthrough(self) -> Model:
if not self.image_uri:
raise ValueError("image_uri is required for pass-through cases")
self.s3_upload_path = None
self.s3_model_data_url = None # <-- add this line
return self._create_model()

**To reproduce**
```
from sagemaker.core.helper.session_helper import Session, get_execution_role
from sagemaker.serve.model_builder import ModelBuilder

sagemaker_session = Session()
region = sagemaker_session.boto_session.region_name
role = get_execution_role(sagemaker_session, use_default=True)
bucket = sagemaker_session.default_bucket()

image_uri = f"763104351884.dkr.ecr.{region}.amazonaws.com/djl-inference:0.33.0-lmi15.0.0-cu128"

model_builder = ModelBuilder(
image_uri=image_uri,
role_arn=role,
sagemaker_session=sagemaker_session,
instance_type="ml.g5.2xlarge",
env_vars={
"HF_MODEL_ID": f"s3://{bucket}/models/any-model",
"OPTION_ROLLING_BATCH": "vllm",
},
)

model_builder.build(model_name="passthrough-bug-repro")

Contributor guide

Open the contributing guide

Research direction

Start at ModelBuilder._build_for_passthrough() and trace the values passed into _create_model(). Reproduce the passthrough build with image_uri and HF_MODEL_ID, then verify that a successful CreateModel request omits ModelDataUrl and the container can load the configured model from S3.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.