aws / aws/sagemaker-python-sdk

New sagemaker.processing.DataWranglerProcessor class

Đang mở
#2,771 1 bình luận 2 reaction 1 người được giao Được @mollyheamazon nhận Xem trên GitHub
component: processing type: feature request
Ngôn ngữ chính
Python
Star
2.3k
Fork
1.3k
Merge trung bình
1 ngày 22 giờ
Pull request đã merge (30 ngày)
35

Mô tả

**Describe the feature you'd like**
Being able to create a Processing Job starting from a SageMaker Data Wrangler Flow template. The Processor should use the Data Wrangler container image. As the template already includes input and output configurations, the DataWranglerProcessor class should map these values to Processing Inputs and Processing Outputs

**How would this feature be used? Please describe.**
For automated data preparation processes built using SageMaker Data Wrangler, I'd use this class to create a Processing Step in the context of a DAG, for example using SageMaker Pipelines, Step Functions which natively support Processing Jobs. And once a flow template is created, it would be easier to move across environments without having to export a Pyhton script from the Data Wrangler console in SageMaker Studio.

**Describe alternatives you've considered**
Currently, the only way to implement automation for a Data Wrangler Flow template is to manually export the process as a Jupyter notebook or Python script, then manually modify the generated script to adjust it to what is needed for the orchestration. Upon any changes in source files or data preparation steps, the output needs to be created again.

**Additional context**
Ideally, in the Processing Script, there should be a logic that reads the provided flow template, parses the inputs and outputs and maps them to corresponding ProcessingInputs and ProcessingOutputs before running the corresponding Processing Job.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.