apache / apache/dolphinscheduler-sdk-python
[impv] Auto create resource when set resource in task if there are exist files in local system
- 主要语言
- Python
- 星标
- 71
- 派生
- 26
- PR 合并指标
- 30 天内没有已合并 PR
描述
```python
# [start workflow]
from pydolphinscheduler.core import Workflow
from pydolphinscheduler.core.resource import Resource
from pydolphinscheduler.tasks import Shell
dependence = "dependence.py"
main = "main.py"
with Workflow(
name="multi_resources_example",
# [start create_new_resources]
resource_list=[
Resource(
name=dependence,
content="from datetime import datetime\nnow = datetime.now()",
),
Resource(name=main, content="from dependence import now\nprint(now)"),
],
# [end create_new_resources]
) as workflow:
# [start use_exists_resources]
task_use_resource = Shell(
name="use-resource",
command=f"python {main}",
resource_list=[
dependence,
main,
],
)
# [end use_exists_resources]
workflow.run()
# [end workflow]
```
Change the above usage to the following usage
```python
# [start workflow]
from pydolphinscheduler.core import Workflow
from pydolphinscheduler.core.resource import Resource, LocalResource
from pydolphinscheduler.tasks import Shell
dependence = "dependence.py"
main = "main.py"
with Workflow(
name="multi_resources_example",
) as workflow:
# [start use_exists_resources]
task_use_resource = Shell(
name="use-resource",
command=f"python {main}",
resource_list=[
LocalResource('dependence.py'),
LocalResource('main.py'),
LocalResource('*.py'),
],
)
# [end use_exists_resources]
workflow.run()
# [end workflow]
```
贡献指南
调研方向
从 Resource 和 LocalResource 入口点以及 issue 中所示的 Shell resource_list 用法开始。跟踪创建任务时如何处理本地资源名称和 glob 模式。完成标准是:无需先定义 Workflow 资源 resource_list,即可通过 LocalResource 提供本地文件,包括 '*.py'。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- api
- Issue 类型
- 功能
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100