apache / apache/beam

[Feature Request]: Allow specifying a TimestampCombiner in BatchElements

Open
#22,853 3 comments 0 reactions 0 assignees View on GitHub
core new feature P2 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

### What would you like to happen?

See https://github.com/apache/beam/pull/22834#discussion_r954117756

Currently BatchElements will always emit with the timestamp at the end of the current window. We should allow users to specify a TimestampCombiner to determine the output timestamp from the input elements.

### Issue Priority

Priority: 2

### Issue Component

Component: sdk-py-core

Contributor guide

Open the contributing guide

Research direction

Start by locating the Python SDK entry point for BatchElements and read the linked pull-request discussion about TimestampCombiner. Trace how BatchElements currently chooses output timestamps, then define completion as allowing a user-specified combiner to determine timestamps from input elements, with coverage for the requested behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.