aws / aws/sagemaker-python-sdk

Enable processing and/or memory optimized instances when using sagemaker.remote_function's @remote decorator

未关闭
#4,040 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
component: training remote-function type: feature request
主要语言
Python
星标
2.3k
派生
1.3k
平均合并
1 天 22 小时
30 天内合并 PR
35

描述

**Describe the feature you'd like**
Currently only training instances are allowed when using @remote (from the sagemaker.remote_function module). This module can be used for processing tasks as well, so it would be useful to have more instance types available.

**How would this feature be used? Please describe.**
Using instances with more than 256GB that don't need GPU acceleration for processing tasks. These are only available as Processing instances as far as I know (and are referred as Memory optimized instances there).

**Describe alternatives you've considered**
We can use Sagemaker Processing jobs, and we currently do that. The downside is that local mode is not enabled when using Sagemaker studio, so it can be a little clunky to develop scripts locally before submitting then to a processing task. This is much easier when using @remote, since we can execute code directly without the need of mapping inputs/outputs, etc. Code for local testing and remote execution could be very similar if not identical in this case.

**Additional context**
In case this is not clear, I'm referring to this functionality: https://docs.aws.amazon.com/sagemaker/latest/dg/train-remote-decorator.html

Link to the instance types available here: https://aws.amazon.com/sagemaker/pricing/

@jmahlik

贡献指南

打开贡献指南

调研方向

从 sagemaker.remote_function 模块及其 @remote 装饰器入手,然后将接受的实例类型与 issue 中链接的 SageMaker remote 装饰器文档进行比较。当 @remote 支持处理任务使用 processing 和 memory-optimized 实例,同时保留现有的训练行为时,即表示完成;如果 repository 中有相关测试,请添加针对性的覆盖。

由索引模型根据 Issue 内容生成。

评估

技术栈
aws, machine-learning, python
领域
cloud, machine-learning
Issue 类型
功能
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
38/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。