StartOfDayMapper does not work on assets that return a Date
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Under which category would you file this issue?
Task SDK
### Apache Airflow version
3.3.0
### What happened and how to reproduce it?
** Issue Description **
StartOfDayMapper defaults input_format to "%Y-%m-%dT%H:%M:%S". If an upstream asset emits a partition key that's already at day granularity (e.g. "2026-07-30"), calling to_downstream() without explicitly overriding input_format raises an error.
As a result, the mapper must be explicitly configured with the correct input_format up front. This prevents flexible handling of a mix of partition keys at different granularities (e.g. some upstream assets emitting day-level keys, others hour-level) without requiring separate, explicitly-configured mapper instances for each.
** Repro **
from airflow.partition_mappers.temporal import StartOfDayMapper
# --- Case 1: full timestamp input (matches the default input_format) ---
mapper = StartOfDayMapper() # default input_format="%Y-%m-%dT%H:%M:%S"
print(mapper.to_downstream("2026-07-30T14:22:05")) # → "2026-07-30"
# --- Case 2: date-only input (works) ---
mapper_date_only = StartOfDayMapper(input_format="%Y-%m-%d")
print(mapper_date_only.to_downstream("2026-07-30")) # → "2026-07-30"
# --- Case 3: date-only input WITHOUT telling it the format (reproduces the error) ---
mapper_default = StartOfDayMapper() # still expects full timestamp
print(
mapper_default.to_downstream("2026-07-30")
) # → ValueError: time data '2026-07-30' does not match format '%Y-%m-%dT%H:%M:%S'
### What you think should happen instead?
Either:
Partition keys already at day granularity should be passed through unchanged, similar to IdentityMapper,
or
StartOfDayMapper should output keys in its own default input_format (i.e. "%Y-%m-%dT%H:%M:%S", with the time component zeroed to T00:00:00), so that its output can be fed back into another instance of the same mapper (chaining).
### Operating System
_No response_
### Deployment
None
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with StartOfDayMapper in airflow.partition_mappers.temporal and run the three reproduction cases from the issue. First confirm whether date-only keys should pass through or be normalized to a zeroed timestamp, then preserve the existing full-timestamp behavior and add regression coverage for the selected contract.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100