apache / apache/dolphinscheduler-sdk-python

[Bug][python] YAML workflow loader cannot resolve task_type "K8S" (Kubernetes.yaml example fails to parse)

Open
#176 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
71
Forks
26
PR merge metrics
No merged PRs in 30d

Description

The shipped example `examples/yaml_define/Kubernetes.yaml` declares `task_type: K8S` but fails to load:

```
pydolphinscheduler.exceptions.PyDSTaskNoFoundException: cant not find task K8S
```

The YAML loader resolves a task type to its task class **by class name only**. In
`src/pydolphinscheduler/core/yaml_workflow.py`, `get_task_cls()` builds
`{type_.capitalize(): type_ for type_ in tasks.__all__}` and looks up
`task_type.capitalize()`. So it expects `"Kubernetes"` (the class name), but the
documented/constant task type is `K8S`:

- `"K8S".capitalize()` → `"K8s"`, which is **not** in the class-name map.
- The internal constant is `TaskType.KUBERNETES = "K8S"` (`constants.py`), i.e. the
rest of the SDK and the backend use `"K8S"`, but the YAML loader never consults
`TaskType` — only class names. The loader and the rest of the SDK disagree on the
K8S identifier.

There is a **second** bug in the same example: it uses camelCase field names
`minCpuCores` / `minMemorySpace`, but the `Kubernetes` task constructor expects
snake_case `min_cpu_cores` / `min_memory_space`. After getting past the loader you
then hit:

```
TypeError: __init__() missing 2 required positional arguments: 'min_cpu_cores' and 'min_memory_space'
```

**Why CI never caught it**

`Kubernetes.yaml` is the only one of the 22 examples **omitted** from the parametrize list in
`tests/core/test_yaml_workflow.py::test_get_create_workflow` (all others are covered).
So the broken example ships untested and CI stays green.

**To Reproduce**

```python
from pydolphinscheduler.core.yaml_workflow import create_workflow
create_workflow("examples/yaml_define/Kubernetes.yaml")
# -> PyDSTaskNoFoundException: cant not find task K8S
```

Or just the resolver:

```python
from pydolphinscheduler.core.yaml_workflow import get_task_cls
get_task_cls("K8S") # -> PyDSTaskNoFoundException: cant not find task K8S
get_task_cls("Kubernetes") # OK (only the class name works today)
```

**Which version are you using?**

Reproduced on:
- **PyPI release `apache-dolphinscheduler` 4.1.0** (latest release).
- **`main` branch HEAD** — `get_task_cls`, `Kubernetes.yaml`, and the parametrize
list are byte-for-byte the same as 4.1.0, confirming this is a current defect
rather than a released-only regression.

**Expected behavior**

`task_type: K8S` (the documented constant value) should resolve to the `Kubernetes`
task class, and the shipped `Kubernetes.yaml` example should parse and be covered by CI.

**Additional context**

I'll open a PR that (1) makes the loader fall back to `TaskType` constant values,
(2) fixes the camelCase fields in the example, and (3) adds the example +
`K8S`/`k8s`/`Kubernetes` cases to the tests.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.