aws / aws/sagemaker-python-sdk

New sagemaker.processing.DataWranglerProcessor class

Aperta
#2,771 1 commento 2 reazioni 1 assegnatario Rivendicata da @mollyheamazon Vedi su GitHub
component: processing type: feature request
Lingua principale
Python
Stelle
2.3k
Fork
1.3k
Merge medio
1g 22h
PR unite (30g)
35

Descrizione

**Describe the feature you'd like**
Being able to create a Processing Job starting from a SageMaker Data Wrangler Flow template. The Processor should use the Data Wrangler container image. As the template already includes input and output configurations, the DataWranglerProcessor class should map these values to Processing Inputs and Processing Outputs

**How would this feature be used? Please describe.**
For automated data preparation processes built using SageMaker Data Wrangler, I'd use this class to create a Processing Step in the context of a DAG, for example using SageMaker Pipelines, Step Functions which natively support Processing Jobs. And once a flow template is created, it would be easier to move across environments without having to export a Pyhton script from the Data Wrangler console in SageMaker Studio.

**Describe alternatives you've considered**
Currently, the only way to implement automation for a Data Wrangler Flow template is to manually export the process as a Jupyter notebook or Python script, then manually modify the generated script to adjust it to what is needed for the orchestration. Upon any changes in source files or data preparation steps, the output needs to be created again.

**Additional context**
Ideally, in the Processing Script, there should be a logic that reads the provided flow template, parses the inputs and outputs and maps them to corresponding ProcessingInputs and ProcessingOutputs before running the corresponding Processing Job.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.