apache / apache/airflow

HITL responses with serde-reserved keys in params_input cannot be resumed or resubmitted

Open
#71,036 0 comments 0 reactions 0 assignees View on GitHub
area:API area:core kind:bug
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### What happened

`airflow.sdk.serde.serialize` refuses any dict carrying its reserved keys (`__classname__`, `__schema_id__`) at any depth. The Human-in-the-loop route stores `params_input` verbatim, with no validation of the key names, so a response whose params contain such a key cannot be serialized when the task is resumed.

Since apache/airflow#70685, `handle_event_submit` no longer raises in that situation — it routes the task instance to `__fail__` so a worker fails it normally, instead of aborting the caller. That is the right trade for the triggerer and the scheduler sweep, where a raise previously wedged an entire batch or caused a poison event to be redelivered indefinitely. It is a regression for this one HITL path:

- **Before:** the request 500s, nothing is recorded, and the user can correct the params and resubmit.
- **After:** the response is recorded, the task is routed to `__fail__`, the route answers `200`, and the `response_received` guard turns a corrected resubmission into a `409`.

The user is left unable to retry a response that was effectively discarded, and the `200` gives them no indication that anything went wrong.

### What needs doing

Validate on the write side, next to the existing option checks in `airflow/api_fastapi/core_api/routes/public/hitl.py`: reject `params_input` containing serde-reserved keys with a `400`, so the request never reaches a state that cannot be resumed. Validating what is accepted is preferable to compensating after it has been stored.

Optionally, the route could also detect the discarded case after the fact — `handle_event_submit` leaves `next_method == TRIGGER_FAIL_REPR` on a task instance it could not resume, which is how the scheduler sweep now distinguishes that outcome — and answer with something other than `200`. That changes a public endpoint's response, so it wants its own discussion.

### Acceptance criteria

- A HITL response whose `params_input` contains `__classname__` or `__schema_id__` is rejected with `400` at submission time.
- The task instance stays in `AWAITING_INPUT` and the user can resubmit a corrected response.
- Test coverage for the rejection and for the resubmission remaining possible.

Raised in review of apache/airflow#70685.

---
Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

Contributor guide

Open the contributing guide

Research direction

In airflow/api_fastapi/core_api/routes/public/hitl.py, start beside the existing option checks and trace how params_input is handled during submission. Add coverage for rejecting serde-reserved keys and for corrected resubmission, then verify the task remains in AWAITING_INPUT and the invalid request returns 400.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.