aws / aws/sagemaker-python-sdk

Environment variables limited to 256 characters

Open
#2,932 4 comments 12 reactions 1 assignee Claimed by @rsareddy0329 View on GitHub
component: pysdk-team type: feature request
Dominant language
Python
Stars
2.3k
Forks
1.3k
Avg merge
1d 22h
Merged PRs (30d)
35

Description

**Describe the bug**
Currently, the value length constraints for environment variables are quite limited in `CreateProcessingJob` and `CreateTrainingJob`.

* [CreateProcessingJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateProcessingJob.html#sagemaker-CreateProcessingJob-request-Environment): 256
* [CreateTrainingJob](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTrainingJob.html#sagemaker-CreateTrainingJob-request-Environment): 512

This causes issues when passing urls or other long strings via the environment parameter, as they often exceed the limit. I am not aware of an alternative to pass items that shouldn't be written to files to a job at runtime. Storing some info on s3 and treating it as an input cannot be used as a workaround, as some variables should not be shared with everyone who has access to the s3 bucket nor stored long term. It's also inconvenient to write a notebook/user's env vars to s3, read them in, set them as env vars etc.

It looks like it would be possible to have an api call accept keys that are over 256 characters in length. Example https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTrainingJob.html#sagemaker-CreateTrainingJob-request-HyperParameters accepts 2500 characters.

**To reproduce**
Pass a string over the char limit similar to below.

```python
estimator = sagemaker.estimator.Estimator(
image,
role,
environment={"LONG_STRING": "".join(("0" for _ in range(513)))},
instance_count=1,
instance_type=instance_type,
)

script_processor = sagemaker.processing.ScriptProcessor(
command=["my", "command"],
env={"LONG_STRING": "".join(("0" for _ in range(257)))},
image_uri=image,
role=role,
instance_count=1,
instance_type=instance_type,
)

```

**Expected behavior**
Ability to pass longer environment variables to jobs.

**Screenshots or logs**

```console
botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreateProcessingJob operation: 1 validation error detected: Value '' at 'environment' failed to satisfy constraint: Map value must satisfy constraint: [Member must have length less than or equal to 256, Member must have length greater than or equal to 0, Member must satisfy regular expression pattern: [\S\s]*]
```

**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: 2.52.0
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: N/A
- **Framework version**: N/A
- **Python version**: 3.9
- **CPU or GPU**: CPU
- **Custom Docker image (Y/N)**: Both

**Additional context**
I suspect others may be facing the same issue as there is mention of passing custom urls for pip configurations.

https://github.com/aws/sagemaker-python-sdk/issues/2207#issuecomment-810096332

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.