Define when the serialized Dag SERIALIZER_VERSION must be bumped
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
- Should wait until https://github.com/apache/airflow/pull/69757#discussion_r3728203900 merged.
### Context
That PR adds an optional per-task `_arg_bindings` property to the serialized Dag. Review split on whether an additive field warrants a `SERIALIZER_VERSION` bump, which surfaced the real gap: there is no written rule for when the version changes, so every PR touching serialization re-litigates it.
### Problem
`SerializedDAG.SERIALIZER_VERSION` is `3` today, and `from_dict` chains `conversion_v1_to_v2` / `conversion_v2_to_v3` for older blobs. But:
- No documented rule says when a bump is required, so "is this additive enough to skip it?" is argued case by case.
- `conversion_v2_to_v3` is a comment plus `ser_obj["__version"] = 3` — it carries no conversion, so the number does not yet signal anything a reader can act on.
- In Airflow 2 the version stayed at `1` for the whole major, so it never carried information. Repeating that in Airflow 3 wastes the mechanism.
- `schema.json` is also weaker than it looks on the write path: `definitions.tasks` is `{"type": "array", "additionalProperties": {...}}`, and `additionalProperties` is inert for arrays, so `validate_schema` never checks task objects against `definitions.operator`. Any policy that leans on "the schema catches it" needs this fixed first.
### Scope
Decide and document:
1. What a bump *means* — a format-compat marker, or a promise that a `conversion_vN_to_vN+1` exists and does real work.
2. Whether additive optional fields bump, or only removals / renames / semantic changes.
3. What a bump obligates: conversion function, `from_dict` accept-list, `schema.json`, tests.
4. How far back `from_dict` supports old versions, and when old conversions can be dropped.
5. Whether to repair the inert `definitions.tasks` validation so `schema.json` is genuinely enforced.
6. Release policy.
7. Date based versioning? Semantic versioning?
Record the outcome in-repo (contributing docs or a `serialized_objects.py` module docstring) so it stops being re-decided per PR.
Contributor guide
Assessment
This issue has not been assessed yet.