apache / apache/airflow

Issue relativedelta midnight serialization

Open Beginner friendly
#70,527 0 comments 0 reactions 0 assignees View on GitHub
area:core area:serialization kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

Airflow 3.x

### What happened and how to reproduce it?

Serializing a `dateutil.relativedelta` that uses absolute time fields set to `0` (e.g. `hour=0` for midnight) drops those fields. After the Dag is deserialized, the schedule no longer snaps to midnight and keeps the base datetime's clock time instead.

Root cause is in `encode_relativedelta` (`airflow-core/src/airflow/serialization/encoders.py`):

```python
encoded = {k: v for k, v in var.__dict__.items() if not k.startswith("_") and v}
```

The `and v` filter treats every falsy value as "unset". That is correct for **relative** fields (`hours`, `days`, …), which default to `0`, but wrong for **absolute** fields (`hour`, `minute`, `second`, `day`, …), which default to `None`. For absolute fields, `0` is meaningful like e.g. `hour=0` means "snap to midnight".

**Minimal repro:**

```python
from datetime import datetime
from dateutil.relativedelta import relativedelta, FR
from airflow.serialization.serialized_objects import BaseSerialization

delta = relativedelta(weekday=FR, hour=0, minute=0, second=0)
base = datetime(2024, 1, 10, 15, 30, 45)

print(base + delta)
# expected: 2024-01-12 00:00:00

round_tripped = BaseSerialization.deserialize(BaseSerialization.serialize(delta))
print(base + round_tripped)
# actual today: 2024-01-12 15:30:45 (hour/minute/second lost)
```

Same bug hits any path that serializes intervals via `encode_relativedelta` / `encode_interval` (Dag schedules, timetables, serialized deltas).

Example Dag schedule that silently changes meaning after parse → serialize → scheduler:

```python
from datetime import datetime
from dateutil.relativedelta import relativedelta, FR
from airflow.sdk import DAG

with DAG(
dag_id="example_friday_midnight",
start_date=datetime(2024, 1, 1),
schedule=relativedelta(weekday=FR, hour=0, minute=0, second=0),
catchup=False,
):
...
```

After serialization, the absolute `hour`/`minute`/`second` fields are gone, so runs are planned at the previous interval's wall-clock time instead of 00:00:00.

### What you think should happen instead?

This is for the example:
Absolute fields with value `0` must survive the encode → decode round-trip. Relative fields that are `0` can still be dropped (they mean "unset").

Expected encode for `relativedelta(weekday=FR, hour=0)`:

```python
{"hour": 0, "weekday": [4]}
```

not only `{"weekday": [4]}`.

### Operating System

Debian

### Deployment

Other

### Apache Airflow Provider(s)

_No response_

### Versions of Apache Airflow Providers

_No response_

### Official Helm Chart version

main (development)

### Kubernetes Version

_No response_

### Helm Chart configuration

_No response_

### Docker Image customizations

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start in airflow-core/src/airflow/serialization/encoders.py at encode_relativedelta, then inspect the BaseSerialization serialize/deserialize path. Reproduce the relativedelta with hour=0, minute=0, and second=0, and verify that absolute zero-valued fields survive the round trip while relative zero fields may still be omitted.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.