aws / aws/sagemaker-python-sdk

Implements PythonScriptProcessor

Đang mở
#4,968 1 bình luận 0 reaction 1 người được giao Được @mollyheamazon nhận Xem trên GitHub
type: feature request
Ngôn ngữ chính
Python
Star
2.3k
Fork
1.3k
Merge trung bình
1 ngày 22 giờ
Pull request đã merge (30 ngày)
35

Mô tả

**Describe the feature you'd like**

I would like to propose a new processor implementation, PythonScriptProcessor. This processor should offer the same features as FrameworkProcessor (such as source_dir support and automatic installation of requirements) but without requiring an Estimator. The goal is to enable users to run Python-based parallel processing jobs more flexibly without coupling them to an estimator.

**How would this feature be used? Please describe.**

This feature would simplify and enhance the experience for users who need to run Python scripts as processing jobs on SageMaker, particularly for use cases that do not require a pre-configured estimator.

Example Use Case:

Running lightweight Python-based data preprocessing or postprocessing scripts on SageMaker Processing jobs.
Performing custom parallel processing tasks, such as batch transformations or distributed computations, where the user’s focus is only on script execution.
The new PythonScriptProcessor would allow users to:

Specify the script directory (source_dir).
Automatically handle dependencies using requirements.txt or other package managers.
Submit a Python script directly for processing without the overhead of setting up an estimator.
This removes the need to over-engineer solutions for straightforward Python tasks and aligns with the existing usability patterns of the SDK.

**Describe alternatives you've considered**

Custom Implementation: Writing custom implementations for dependency handling and script execution within a processing job is repetitive, error-prone, and not aligned with the SageMaker Python SDK's existing abstraction patterns.

**Additional context**

The introduction of PythonScriptProcessor would fill the usability gap between ScriptProcessor and FrameworkProcessor, providing a clean and consistent user experience for Python-based processing tasks. It would also reduce friction for users performing data preparation, transformation, or parallel tasks that don’t require model estimators.

This enhancement would make the SDK more intuitive and better suited for broader workflows in SageMaker.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.