apache / apache/airflow

ParamValidationError are not being displayed in TriggerDagRunOperator task logs

Open
#58,065 7 comments 2 reactions 0 assignees View on GitHub
area:core area:Triggerer kind:bug
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Apache Airflow version

Other Airflow 2/3 version (please specify below)

### If "Other Airflow 2/3 version" selected, which one?

3.1.1

### What happened?

When using `TriggerDagRunOperator`, `airflow.exceptions.ParamValidationError` are not being shown in the triggerer log.
In the triggerer (task) log, this is what we see:
```
AirflowRuntimeError: API_SERVER_ERROR: {'status_code': 500, 'message': 'Server returned error', 'detail': {'message': 'Internal server error', 'correlation-id': '...'}}
```

The true error is only unveiled when we dig into the api-server logs.

### What you think should happen instead?

`airflow.exceptions.ParamValidationError` should show up in the log of `TriggerDagRunOperator` tasks.

### How to reproduce

1. Build a DAG `underlying_dag` that takes certain Params, e.g. `"test_param": Param("", type="string", enum=["a", "b"]),`
2. use TriggerDagRunOperator in a trigger DAG `trigger_dag`
3. call the TriggerDagRunOperator with an invalid parameter, e.g.
```
TriggerDagRunOperator.partial(task_id="trigger_dag", trigger_dag_id="underlying_dag").expand(
conf=[{"test_param": "d"}]
)
```
4. observe that the task fails and the logging described above.
5. observe the ParamValidationError in api-server

### Operating System

astro managed docker image, not sure what OS, debian?

### Versions of Apache Airflow Providers

_No response_

### Deployment

Astronomer

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start with the TriggerDagRunOperator reproduction described in the issue and compare its task log with the api-server log when an invalid Param is supplied. Trace how ParamValidationError becomes AirflowRuntimeError, then add coverage showing the validation error is exposed in the operator task log.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.