Provide feedback when accessing undeclared outlet at runtime
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
Currently, operations toward a non-declared outlet to `outlet_events` is simply ignored:
```python
@task
def emit_alias_event():
context = get_current_context()
context["outlet_events"][AssetAlias("a")].add(Asset("new"))
# Line above does nothing since AssetAlias("a") is not an outlet.
```
The same also applies to `yield Metadata`.
This is a bit confusing to the user since they did something, and reasonably expect it’s an Airflow bug if nothing seems to happen.
I can think of a couple of ways to resolve this:
#### Automatically add outlets when you do this
If you access something, it automatically becomes an outlet. However, Airflow would not be able to detect such a dynamic outlet in many other places, such as the dag dependency graph (which is generated statically). Would this only shift the confusion elsewhere?
#### Raise a KeyError
Or a more detailed error with a hint. More explicit, but would it be awkward for certain workflows?
### Use case/motivation
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the runtime handling of context["outlet_events"] and yielded Metadata in the task example. Decide between automatic outlet registration and raising a KeyError or more detailed error, then define the expected behavior for undeclared outlets and verify it in the relevant runtime tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100