Typing error
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
3.0.3
### If "Other Airflow 2 version" selected, which one?
3.0.2
### What happened?
I wrote a custom deferrable Operator, that uses a custom trigger for deferring. The custom trigger should fail/succeed the task (yielding `TaskFailedEvent` or `TaskSuccessEvent`).
According to the [docstrings](https://github.com/apache/airflow/blob/8fb611667e83ac037ee16722549efedef7350ee7/task-sdk/src/airflow/sdk/bases/operator.py#L1586), I should pass `method_name=None` in this case. However, that leads to a typing issue because `method_name` has type `string `rather than `Optional(string)`
Indeed, the task fails with a validation error:
```
- Top level error: source="task"
ValidationError: 1 validation error for DeferTask
next_method
Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]
```
### What you think should happen instead?
When passing `None` as `next_method`, the task should be failed/succeeded directly from the trigger.
### How to reproduce
Define a custom deferrable Operator, that uses a custom trigger for deferring. The custom trigger should only fail/succeed the task (yielding TaskFailedEvent or TaskSuccessEvent).
Define the custom operator's `execute` function as :
```
def execute(self):
self.defer(
trigger=CustomTrigger(),
method_name=None, # type:ignore
# according to docs:
# If the trigger will end the task instance itself,
# ``method_name`` should be None
# But it's not nullable
)
```
And use the custom operator as a single task in a dag
### Operating System
RedHat Openshift
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start in task-sdk/src/airflow/sdk/bases/operator.py, reading the defer method and its method_name docstring. Trace the DeferTask validation reported in the issue and reproduce the custom deferrable operator with method_name=None. Done means the value is accepted and a trigger yielding TaskFailedEvent or TaskSuccessEvent can complete the task directly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100