apache / apache/beam

Record type of state in Python InMemoryUnmergedState

Open
#18,441 0 comments 0 reactions 0 assignees View on GitHub
bug core P3 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

Currently, the Python InMemoryUnmergedState implementation does not record the type of tag used to store state. This means that, for example, it is hard to enumerate all state of a specific type of tag (e.g. WatermarkHolds). We should fix this so that we can more cleanly extract, for example, the earliest WatermarkHold for the state of a given key.

Imported from Jira [BEAM-2519](https://issues.apache.org/jira/browse/BEAM-2519). Original Jira may contain additional context.
Reported by: ccy.

Contributor guide

Open the contributing guide

Research direction

Start by locating the Python InMemoryUnmergedState implementation and inspect how state is stored for each key and tag. The change is complete when the implementation records each tag's type and callers can enumerate state of a selected type, such as WatermarkHolds, to identify the earliest matching hold.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.