apache / apache/airflow

Deferrable HttpSensor does not move to Triggerer when using `response_check`

Open
#40,209 6 comments 0 reactions 1 assignee Assigned to @dabla View on GitHub
good first issue kind:bug needs-triage provider:http
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Discussed in https://github.com/apache/airflow/discussions/39597

Originally posted by **bejota** May 13, 2024
Hello,
I'm using HttpSensor to poll SLURM's REST API for job status. I'm using `deferrable` mode with `response_check`. The task only goes to the triggerer if I remove the `response_check`.

Airflow Docker Image `apache/airflow:latest-python3.9`
Airflow Version `2.9.0`
apache-airflow-providers-http `4.10.0`

Does not move to triggerer:
```
check_slurm_job = HttpSensor(
task_id="check_slurm_job",
http_conn_id="slurm_conn_id",
method="get",
endpoint="slurm/v0.0.40/job/999",
response_check=lambda response: "COMPLETED" in response.text,
poke_interval=10,
timeout=60 * 60 * 5,
deferrable=True, # not deferring to triggerer ?
dag=dag,
)
```
Moves to triggerer:
```
check_slurm_job = HttpSensor(
task_id="check_slurm_job",
http_conn_id="slurm_conn_id",
method="get",
endpoint="slurm/v0.0.40/job/999",
# response_check=lambda response: "COMPLETED" in response.text,
poke_interval=10,
timeout=60 * 60 * 5,
deferrable=True, # not deferring to triggerer ?
dag=dag,
)
```
In digging further, I see that HttpTrigger has no `response_check` method. I will attempt to add one, unless there is a better approach to this problem?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.