aws / aws/sagemaker-python-sdk

Pipeline parameters different behavior in comparison to v2

未关闭
#5,614 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
component: pipelines type: bug
主要语言
Python
星标
2.3k
派生
1.3k
平均合并
1 天 22 小时
30 天内合并 PR
35

描述

**PySDK Version**
- [ ] PySDK V2 (2.x)
- [x] PySDK V3 (3.x)

**Describe the bug**
In v2, it was possible to pass pipeline variables to environment variables into an `Estimator`. For example, an environment variable "RANDOM_STATE" could be set to a value given as a pipeline variable. This does not work in v3 in a `ModelTrainer` instance, resulting in a validation error: `ValidationError: 1 validation error for ModelTrainer`. A similar issue exists in the `HyperparameterTuner` which does not accept a pipeline variable for the `random_seed`, returning `ValidationError: 1 validation error for HyperParameterTuningJobConfig`.

**To reproduce**
Create `ModelTrainer` instance and pass a pipeline parameter into the `environment` argument dictionary.
Create a `HyperparameterTuner` instance and pass a pipeline parameter into the `random_seed` argument.
```
model = ModelTrainer(
source_code=source_code,
compute=compute,
networking=self.networking,
base_job_name=base_job_name,
training_image=self.image_uris["train"],
output_data_config=OutputDataConfig(
s3_output_path=Join(
on="/",
values=[
self.s3_uri_runtime,
ExecutionVariables.PIPELINE_EXECUTION_ID,
"02_mt_output",
],
),
kms_key_id=self.aws_params["kms_key_hub"],
),
stopping_condition=StoppingCondition(max_runtime_in_seconds=28800),
role=self.aws_params["exec_role"],
sagemaker_session=self.pipeline_session,
environment={
"RANDOM_STATE": self.pipeline_params["RandomState"].to_string(), # <-- This line causes issues
**self.default_env_vars,
},
)

hyperparameter_tuner = HyperparameterTuner(
model_trainer=model,
base_tuning_job_name=base_job_name,
metric_definitions=metric_definitions,
objective_metric_name=self.hpt_params["objective_metric_name"],
objective_type=self.hpt_params["objective_type"],
hyperparameter_ranges=self.hpt_params["hyperparameter_ranges"],
max_jobs=self.hpt_params["max_jobs"],
strategy="Bayesian",
max_parallel_jobs=4,
random_seed=self.pipeline_params["RandomState"], # <-- This line causes issues
tags=self.tags,
)
```

**Expected behavior**
Pipeline variables should be able to affect environment variables, as well as the `random_seed` argument of the `HyperparameterTuner`.

**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: 3.5.0

**Additional context**
This is a roadblock for us regarding a migration from v2 to v3.

贡献指南

打开贡献指南

调研方向

首先,使用 SageMaker Python SDK 3.5.0 和所示的 pipeline 参数,复现 ModelTrainer(environment=...) 和 HyperparameterTuner(random_seed=...) 中的验证错误。将这些路径与 issue 中描述的 v2 行为进行比较。当 pipeline 变量可以在没有验证错误的情况下同时提供 RANDOM_STATE 环境值和 random_seed 时,即视为完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
aws, python
领域
machine-learning
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
38/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。