aws / aws/sagemaker-python-sdk
FrameworkProcessor.run(requirements=...) ignores custom requirements file in PySDK V3
- Ngôn ngữ chính
- Python
- Star
- 2.3k
- Fork
- 1.3k
- Merge trung bình
- 1 ngày 22 giờ
- Pull request đã merge (30 ngày)
- 35
Mô tả
**PySDK Version**
- [ ] PySDK V2 (2.x)
- [X] PySDK V3 (3.x)
**Describe the bug**
FrameworkProcessor.run(requirements=...) in the SageMaker v3 sagemaker-core Processing API accepts a custom requirements file name, but the generated runtime script does not use it.
For example, this API call accepts:
framework_processor.run(
code="run_entrypoint.py",
source_dir=".",
requirements="cpu-requirements.txt",
)
However, the generated runproc.sh only checks for a hardcoded file named requirements.txt:
```
if [[ -f 'requirements.txt' ]]; then
pip uninstall --yes typing
pip install -r requirements.txt
fi
```
So cpu-requirements.txt may be packaged into sourcedir.tar.gz, but it is not installed at runtime.
Relevant file:
sagemaker-core/src/sagemaker/core/processing.py
Relevant method:
FrameworkProcessor._generate_framework_script()
Reference:
https://github.com/aws/sagemaker-python-sdk/blob/master/sagemaker-core/src/sagemaker/core/processing.py#L1187
To reproduce
Create the following project structure:
.
├── run_entrypoint.py
├── cpu-requirements.txt
└── scripts/
└── entrypoints/
└── step_preprocess_jobs.py
Example cpu-requirements.txt:
requests==2.32.3
Example run_entrypoint.py:
```
import importlib
import sys
module_name = sys.argv[1]
module = importlib.import_module(module_name)
if hasattr(module, "main"):
module.main()
```
Example scripts/entrypoints/step_preprocess_jobs.py:
```
def main():
import requests
print("requests imported successfully")
```
Then create a SageMaker Processing step using FrameworkProcessor.run(...) with a custom requirements file name:
```
from sagemaker.core.processing import FrameworkProcessor
from sagemaker.core.workflow.pipeline_context import PipelineSession
from sagemaker.core.shapes import ProcessingInput, ProcessingOutput, ProcessingS3Input, ProcessingS3Output
role = ""
image_uri = ""
bucket = ""
pipeline_session = PipelineSession()
framework_processor = FrameworkProcessor(
image_uri=image_uri,
role=role,
instance_count=1,
instance_type="ml.m5.large",
sagemaker_session=pipeline_session,
)
step_args = framework_processor.run(
code="run_entrypoint.py",
source_dir=".",
requirements="cpu-requirements.txt",
arguments=["scripts.entrypoints.step_preprocess_jobs"],
inputs=[
ProcessingInput(
input_name="dummy_input",
s3_input=ProcessingS3Input(
s3_uri=f"s3://{bucket}/dummy-input/",
local_path="/opt/ml/processing/input/dummy",
s3_data_type="S3Prefix",
s3_input_mode="File",
),
)
],
outputs=[
ProcessingOutput(
output_name="dummy_output",
s3_output=ProcessingS3Output(
s3_uri=f"s3://{bucket}/dummy-output/",
local_path="/opt/ml/processing/output",
s3_upload_mode="EndOfJob",
),
)
],
)
```
The generated processing runtime script only installs requirements.txt, not the file passed through the requirements argument.
The relevant generated script logic is:
if [[ -f 'requirements.txt' ]]; then
pip uninstall --yes typing
pip install -r requirements.txt
fi
As a result, dependencies listed in cpu-requirements.txt are not installed.
Expected behavior
When a user passes:
requirements="cpu-requirements.txt"
to FrameworkProcessor.run(...), the generated runtime script should install dependencies from that file:
pip install -r cpu-requirements.txt
The requirements argument should be respected.
For backward compatibility, if requirements is not provided, the implementation may continue checking for and installing from requirements.txt.
Current generated script behavior:
```
cd /opt/ml/processing/input/code/
if [ -f sourcedir.tar.gz ]; then
tar -xzf sourcedir.tar.gz
else
echo "ERROR: sourcedir.tar.gz not found!"
exit 1
fi
if [[ -f 'requirements.txt' ]]; then
pip uninstall --yes typing
pip install -r requirements.txt
fi
```
Expected generated script behavior when requirements="cpu-requirements.txt" is provided:
```
if [[ -f 'cpu-requirements.txt' ]]; then
pip uninstall --yes typing
pip install -r cpu-requirements.txt
else
echo "WARNING: requirements file not found: cpu-requirements.txt"
fi
```
System information
* SageMaker Python SDK version: PySDK V3 / sagemaker-core
* Framework name (eg. PyTorch) or algorithm (eg. KMeans): FrameworkProcessor
* Framework version: N/A
* Python version: N/A
* CPU or GPU: CPU
* Custom Docker image (Y/N): Y
Additional context
The requirements parameter is accepted by FrameworkProcessor.run(...) and passed into internal methods such as _pack_and_upload_code(...) and _package_code(...), but it is not used when generating the runtime shell script.
The root cause appears to be that _generate_framework_script(...) hardcodes requirements.txt instead of using the requirements argument passed by the user.
I would like to work on this issue and submit a PR to fix the behavior.
Hướng dẫn đóng góp
Hướng nghiên cứu
Bắt đầu trong sagemaker-core/src/sagemaker/core/processing.py tại FrameworkProcessor._generate_framework_script(), sau đó theo dõi cách requirements được truyền qua _pack_and_upload_code() và _package_code(). Tái hiện bằng ví dụ FrameworkProcessor.run được cung cấp và kiểm tra runtime script đã tạo; hoàn tất khi một tên tệp requirements tùy chỉnh được sử dụng, trong khi hành vi mặc định vẫn được hỗ trợ.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- aws, python, shell
- Lĩnh vực
- api, machine-learning
- Loại issue
- Lỗi
- Độ khó
- 3/5
- Thời gian dự kiến
- 1-2 ngày
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức phù hợp với người mới
- 74/100