apache / apache/beam

outputWithTimestamp() accepts timestamps that will fail preconditions

Open
#18,188 0 comments 0 reactions 0 assignees View on GitHub
core improvement java P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

We have accidentally created events with *wrong* timestamps in the future which are accepted by
outputWithTimestamp(), but will fail at a later step:

java.lang.IllegalStateException: Timer 472976-06-15T20:09:57.269Z is beyond end-of-time
at Preconditions.checkState(Preconditions.java:199)
at ReduceFnRunner.scheduleEndOfWindowOrGarbageCollectionTimer(ReduceFnRunner.java:1050)
[...]

Would it make sense to implement a check already at outputWithTimestamp() level to fail early?

Imported from Jira [BEAM-1408](https://issues.apache.org/jira/browse/BEAM-1408). Original Jira may contain additional context.
Reported by: andyxu.

Contributor guide

Open the contributing guide

Research direction

Start by locating outputWithTimestamp() and tracing how its timestamp reaches ReduceFnRunner.scheduleEndOfWindowOrGarbageCollectionTimer(). Review the preconditions in Preconditions.java and the reported beyond-end-of-time failure; done means invalid future timestamps fail at outputWithTimestamp() rather than at the later timer step.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
stream-processing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.