Unexpected primitive based Enum (de)serialization
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Apache Airflow version
main (development)
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
When enumeration inherits some primitive type (e.g. str, int), deserialization makes the value primitive as well.
```python
import enum
from airflow.serialization.serde import deserialize, serialize
class MyEnum(enum.IntEnum):
A = 1
B = 2
value = MyEnum.A
res = deserialize(serialize(value))
assert res is MyEnum.A, f'{type(res)=}, {res=}'
# AssertionError: type(res)=, res=1
```
### What you think should happen instead?
This is little unexpected for me as a user and for mypy
```python
@task
def my_task(v: MyEnum):
assert is instance(v, MyEnum), 'fine for mypy, fails in runtime'
v = MyEnum(v)
...
my_task(MyEnum.A)
```
The other issue is that if I add airflow compatible custom serialization the result is incorrect too
```python
class MyEnum(enum.IntEnum):
A = 1
B = 2
def serialize(self):
return f'{self.name}.{self.value}'
res = serialize(MyEnum.A)
assert res == 'A.1', f'{type(res)=}, {res=}'
# AssertionError: type(res)=, res=1
```
I would expect one of
- implement default serialization and deserialization for enums
- fail with `TypeError: cannot serialize object of type ` unless it implements the interface and is registered for deserialization (as it happens for enums that do not inherit primitive)
### How to reproduce
See the snippet above
### Operating System
Debian GNU/Linux 12 (bookworm)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the serialize and deserialize entry points in airflow.serialization.serde and reproduce the IntEnum examples from the issue. Resolve which behavior is intended for primitive-based enums, then verify that serialization preserves the enum type or consistently rejects unsupported custom serialization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100