apache / apache/airflow

serde: uuid and datetime serializers accept payloads from a newer version

Open Beginner friendly
#72,636 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Apache Airflow version

main (3.4.0, `123b1be658`)

### What happened and how to reproduce it?

Registered serde serializers are expected to reject a payload whose recorded version is newer than the version of the class doing the deserializing — that is what lets an older Airflow refuse an XCom written by a newer one instead of misreading it. Ten of the twelve serializers in `task-sdk/src/airflow/sdk/serde/serializers/` do this. `uuid.py` and `datetime.py` do not.

The framework does not cover for them. `serde/__init__.py` dispatches a registered deserializer directly:

```python
if classname in _deserializers:
return _deserializers[classname].deserialize(cls, version, deserialize(value))
```

Its own `version > class_version` check further down applies only to the attr/dataclass fallback, which a registered serializer never reaches. So for these two the check is simply absent.

```python
from decimal import Decimal
import uuid as uuid_mod, datetime as dt
from airflow.sdk.serde.serializers import uuid as s_uuid, datetime as s_dt, bignum

bignum.deserialize(Decimal, 99, "1.5") # TypeError, as intended
s_uuid.deserialize(uuid_mod.UUID, 99, "12345678-1234-5678-1234-567812345678")
s_dt.deserialize(dt.timedelta, 99, 60.0)
```

On `main`:

```
declared __version__: uuid=1 datetime=2 bignum=1 numpy=1

deserializing a version=99 payload:
bignum (guarded) rejected (serialized 99 of decimal.Decimal > 1)
uuid (no guard) ACCEPTED -> UUID('12345678-1234-5678-1234-567812345678')
datetime (no guard) ACCEPTED -> datetime.timedelta(seconds=60)
```

`kubernetes.py` has no `deserialize` at all, so the remaining eleven are the relevant set and only these two are missing it.

### What you think should happen instead?

Both should raise the same way their siblings do, so that a payload from a newer Airflow fails loudly rather than being read under assumptions that no longer hold.

There is no impact today — both serializers are at the version they ship with, so no payload in the wild carries a higher one. This is about the next time either format changes: whoever bumps `__version__` would reasonably expect the guard to already be there, and its absence would not be obvious from the diff.

### Anything else?

The two files are the only ones where the check is missing, so the fix is to add the same guard the other nine already use. No behaviour changes for any payload that exists now.

### Are you willing to submit PR?

Yes — PR to follow.

Contributor guide

Open the contributing guide

Research direction

Start with task-sdk/src/airflow/sdk/serde/serializers/uuid.py and datetime.py, then compare their deserialization paths with the guarded sibling serializers. Read the registered-deserializer dispatch in serde/__init__.py. Done means newer recorded versions are rejected like bignum while current payloads retain their existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.