apache / apache/dolphinscheduler-sdk-python

[impv] Auto create resource when set resource in task if there are exist files in local system

Aperta
#58 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
71
Fork
26
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

```python
# [start workflow]
from pydolphinscheduler.core import Workflow
from pydolphinscheduler.core.resource import Resource
from pydolphinscheduler.tasks import Shell

dependence = "dependence.py"
main = "main.py"

with Workflow(
name="multi_resources_example",
# [start create_new_resources]
resource_list=[
Resource(
name=dependence,
content="from datetime import datetime\nnow = datetime.now()",
),
Resource(name=main, content="from dependence import now\nprint(now)"),
],
# [end create_new_resources]
) as workflow:
# [start use_exists_resources]
task_use_resource = Shell(
name="use-resource",
command=f"python {main}",
resource_list=[
dependence,
main,
],
)
# [end use_exists_resources]

workflow.run()
# [end workflow]
```
Change the above usage to the following usage
```python
# [start workflow]
from pydolphinscheduler.core import Workflow
from pydolphinscheduler.core.resource import Resource, LocalResource
from pydolphinscheduler.tasks import Shell

dependence = "dependence.py"
main = "main.py"

with Workflow(
name="multi_resources_example",
) as workflow:
# [start use_exists_resources]
task_use_resource = Shell(
name="use-resource",
command=f"python {main}",
resource_list=[
LocalResource('dependence.py'),
LocalResource('main.py'),
LocalResource('*.py'),
],
)
# [end use_exists_resources]

workflow.run()
# [end workflow]
```

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start with the Resource and LocalResource entry points and the Shell resource_list usage shown in the issue. Trace how local resource names and glob patterns are handled when a task is created. Done means local files can be supplied through LocalResource, including '*.py', without defining a Workflow resource_list first.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
api
Tipo di issue
Funzionalità
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.