apache / apache/dolphinscheduler-sdk-python
[Bug][python] YAML workflow loader cannot resolve task_type "K8S" (Kubernetes.yaml example fails to parse)
- Dominant language
- Python
- Stars
- 71
- Forks
- 26
- PR merge metrics
- No merged PRs in 30d
Description
The shipped example `examples/yaml_define/Kubernetes.yaml` declares `task_type: K8S` but fails to load:
```
pydolphinscheduler.exceptions.PyDSTaskNoFoundException: cant not find task K8S
```
The YAML loader resolves a task type to its task class **by class name only**. In
`src/pydolphinscheduler/core/yaml_workflow.py`, `get_task_cls()` builds
`{type_.capitalize(): type_ for type_ in tasks.__all__}` and looks up
`task_type.capitalize()`. So it expects `"Kubernetes"` (the class name), but the
documented/constant task type is `K8S`:
- `"K8S".capitalize()` → `"K8s"`, which is **not** in the class-name map.
- The internal constant is `TaskType.KUBERNETES = "K8S"` (`constants.py`), i.e. the
rest of the SDK and the backend use `"K8S"`, but the YAML loader never consults
`TaskType` — only class names. The loader and the rest of the SDK disagree on the
K8S identifier.
There is a **second** bug in the same example: it uses camelCase field names
`minCpuCores` / `minMemorySpace`, but the `Kubernetes` task constructor expects
snake_case `min_cpu_cores` / `min_memory_space`. After getting past the loader you
then hit:
```
TypeError: __init__() missing 2 required positional arguments: 'min_cpu_cores' and 'min_memory_space'
```
**Why CI never caught it**
`Kubernetes.yaml` is the only one of the 22 examples **omitted** from the parametrize list in
`tests/core/test_yaml_workflow.py::test_get_create_workflow` (all others are covered).
So the broken example ships untested and CI stays green.
**To Reproduce**
```python
from pydolphinscheduler.core.yaml_workflow import create_workflow
create_workflow("examples/yaml_define/Kubernetes.yaml")
# -> PyDSTaskNoFoundException: cant not find task K8S
```
Or just the resolver:
```python
from pydolphinscheduler.core.yaml_workflow import get_task_cls
get_task_cls("K8S") # -> PyDSTaskNoFoundException: cant not find task K8S
get_task_cls("Kubernetes") # OK (only the class name works today)
```
**Which version are you using?**
Reproduced on:
- **PyPI release `apache-dolphinscheduler` 4.1.0** (latest release).
- **`main` branch HEAD** — `get_task_cls`, `Kubernetes.yaml`, and the parametrize
list are byte-for-byte the same as 4.1.0, confirming this is a current defect
rather than a released-only regression.
**Expected behavior**
`task_type: K8S` (the documented constant value) should resolve to the `Kubernetes`
task class, and the shipped `Kubernetes.yaml` example should parse and be covered by CI.
**Additional context**
I'll open a PR that (1) makes the loader fall back to `TaskType` constant values,
(2) fixes the camelCase fields in the example, and (3) adds the example +
`K8S`/`k8s`/`Kubernetes` cases to the tests.
Contributor guide
Assessment
This issue has not been assessed yet.