apache / apache/airflow

unclear docs on custom decorators

Open
#33,167 11 comments 0 reactions 1 assignee Claimed by @rsdpyenugula View on GitHub
contributors-workshop good first issue kind:documentation
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### What do you see as an issue?

Referring to: https://airflow.apache.org/docs/apache-airflow/2.3.4/howto/create-custom-decorator.html

The documentation is unclear. It assumes that the reader has done a bunch of things that haven't been well defined and then the last section talks about the ProviderManager class but it's very brief. You're expected to already know what the ProviderManager is. If I use the search box in the docs then apparently ProviderManager is only mentioned in the create-custom-decorator page.

It seems like the only way to make use of this feature is to go digging through the source code. The operator decorator code is not the most straight-forward to read. I assume that most people wont be able to make use of this feature at all.

### Solving the problem

Either turn it into a tutorial and walk the reader through the whole process in another way.

### Anything else

This is a workaround that seems to do the trick:

I initially came across this because I wanted to make a django operator that would work with the taskflow api. I did this and it seems to work.

```
def make_django_decorator(app_path, settings_module):
def django_task(task_id: str = None, *args, **kwargs):
def django_task_decorator(fn):
@task(task_id=task_id or fn.__name__, *args, **kwargs)
def new_fn(*args, **kwargs):
django_connect(app_path, settings_module) # connect to the database. Now fn can make use of Django models
return fn(
*args,
)

return new_fn

return django_task_decorator

return django_task

# I have a Django project named Tilde, now I have a tilde-specific task decorator. I should be able to make operatrs for other Django projects f I want to

tilde_task = make_django_decorator(
app_path=TILDE_REPO_PATH / "backend", settings_module="backend.settings"
)

```

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.