apache / apache/dolphinscheduler-sdk-python

[impv] Auto create resource when set resource in task if there are exist files in local system

未关闭
#58 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
71
派生
26
PR 合并指标
30 天内没有已合并 PR

描述

```python
# [start workflow]
from pydolphinscheduler.core import Workflow
from pydolphinscheduler.core.resource import Resource
from pydolphinscheduler.tasks import Shell

dependence = "dependence.py"
main = "main.py"

with Workflow(
name="multi_resources_example",
# [start create_new_resources]
resource_list=[
Resource(
name=dependence,
content="from datetime import datetime\nnow = datetime.now()",
),
Resource(name=main, content="from dependence import now\nprint(now)"),
],
# [end create_new_resources]
) as workflow:
# [start use_exists_resources]
task_use_resource = Shell(
name="use-resource",
command=f"python {main}",
resource_list=[
dependence,
main,
],
)
# [end use_exists_resources]

workflow.run()
# [end workflow]
```
Change the above usage to the following usage
```python
# [start workflow]
from pydolphinscheduler.core import Workflow
from pydolphinscheduler.core.resource import Resource, LocalResource
from pydolphinscheduler.tasks import Shell

dependence = "dependence.py"
main = "main.py"

with Workflow(
name="multi_resources_example",
) as workflow:
# [start use_exists_resources]
task_use_resource = Shell(
name="use-resource",
command=f"python {main}",
resource_list=[
LocalResource('dependence.py'),
LocalResource('main.py'),
LocalResource('*.py'),
],
)
# [end use_exists_resources]

workflow.run()
# [end workflow]
```

贡献指南

打开贡献指南

调研方向

从 Resource 和 LocalResource 入口点以及 issue 中所示的 Shell resource_list 用法开始。跟踪创建任务时如何处理本地资源名称和 glob 模式。完成标准是:无需先定义 Workflow 资源 resource_list,即可通过 LocalResource 提供本地文件,包括 '*.py'。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
api
Issue 类型
功能
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。