aws / aws/sagemaker-python-sdk

JumpStartModel corrupts environment variables (incorrect snake_case conversion) when using a private hub

Open
#6,191 0 comments 0 reactions 0 assignees View on GitHub
component: jumpstart type: bug
Dominant language
Python
Stars
2.3k
Forks
1.3k
Avg merge
2d 2h
Merged PRs (30d)
32

Description

**PySDK Version**

* [x] PySDK V2 (2.x)
* [x] PySDK V3 (3.x)

**Describe the bug**
When using the `JumpStartModel` class to deploy a model reference hosted on a private hub (by passing the `hub_name` parameter), a subset of the container environment variable names arrive corrupted. The variable names from the instance-specific overrides are recursively converted from UpperCamelCase to snake_case. The SDK uses a regular expression that inserts an underscore before every capital letter and then lowercases the result.

As a result, a variable like `SM_VLLM_MAX_MODEL_LEN` becomes `s_m__v_l_l_m__m_a_x__m_o_d_e_l__l_e_n`. This causes the instance-specific override to silently fail and not merge over the base defaults, as the spelling collision is broken. The defect is located in the hub deserialization path (`JumpStartInstanceTypeVariants.from_describe_hub_content_response`), which applies a `walk_and_apply_json(response, camel_to_snake)` function across the entire document structure.

**To reproduce**

```python
from sagemaker.jumpstart.model import JumpStartModel

# Note: Requires a private hub setup with a ModelReference to a JumpStart model
model = JumpStartModel(
model_id="huggingface-vlm-gemma-4-e4b-it",
model_version="*",
config_name="generate_lowest_cost",
hub_name="mi-hub-jumpstart-2", # Replace with your private hub name
region="eu-south-2",
)

# Inspect the container definition before creating the model
container_def = model.prepare_container_def(
accept_eula=True
)

# Check the environment variables
print(container_def["Environment"])

```

The environment variables will already show up as corrupted when executing this method.

**Expected behavior**
The environment variables from instance-specific overrides (located under `HostingInstanceTypeVariants`) should retain their original names (e.g., `SM_VLLM_MAX_MODEL_LEN`). The values from these instance-specific overrides should properly override the base defaults without having their keys modified into snake_case with underscores during JSON parsing.

**Screenshots or logs**
Example of the corrupted environment block showing duplicated values:

```text
SM_VLLM_MAX_MODEL_LEN = 4096
s_m__v_l_l_m__m_a_x__m_o_d_e_l__l_e_n = 131072
h_f__h_u_b__o_f_f_l_i_n_e = 1

```

**System information**

* **SageMaker Python SDK version**: 2.245.0 (Defect confirmed by AWS Support to also be present in recent versions 2.257.6 and 3.20.0)

* **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: JumpStart

* **Framework version**: N/A
* **Python version**: Python 3

* **CPU or GPU**: Any

* **Custom Docker image (Y/N)**: N

**Additional context**
AWS Premium Support investigated this under Case ID 178704468000307. They confirmed that the equivalent public-hub function, `from_json`, performs no key conversion at all, which is why the exact same model deploys correctly when taken from the public catalog. The issue occurs specifically because the names in `HostingInstanceTypeVariants` are defined as keys in a map, and are thus incorrectly caught by the `camel_to_snake` recursive rewrite during hub deserialization.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.