[Connector] Expand AWS SageMaker managed job support
- Dominant language
- Go
- Stars
- 7.5k
- Forks
- 886
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 120
Description
## Summary
Expand `flytekitplugins-awssagemaker` beyond model deployment and endpoint management with first-class async connectors for SageMaker managed jobs.
## Proposed scope
- Training jobs
- Processing jobs
- Batch Transform jobs
- Hyperparameter Tuning jobs
- Inference Recommender jobs
Each task follows Flyte's async connector contract:
- `create()` submits the corresponding SageMaker job
- `get()` polls the describe API and maps SageMaker states to Flyte phases
- `delete()` stops the job idempotently
- terminal responses are projected into stable, downstream-friendly result dictionaries
The implementation reuses the plugin's existing boto3 configuration templating (`{inputs.X}`, `{images.X}`, and `{idempotence_token}`), requires no FlytePropeller changes, and registers each connector through the existing `flytekit.plugins` entry-point mechanism.
## Follow-up
A separate follow-up will add Flyte-native Pythonic execution for Training and Processing, where a normal `@task` function body runs inside the SageMaker container through `ContainerEntrypoint`. Keeping that execution model separate allows the conventional boto3/config connector surface to be reviewed and merged independently.
## Validation
- Full SageMaker plugin unit suite
- Standard flytekit pre-commit checks (`ruff`, formatting, codespell, pydoclint)
- Connector registration/import validation
- AWS smoke validation for the managed job lifecycle
Contributor guide
Research direction
Start by reading the existing model-deployment and endpoint-management connectors in flytekitplugins-awssagemaker, then inspect the flytekit.plugins entry-point registration and the plugin unit suite. Done means all five managed job types support create(), get(), and delete(), stable terminal results, registration validation, passing pre-commit checks, and successful AWS lifecycle smoke validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100