apache / apache/beam

Extension for sketch-based statistics

Open
#18,488 0 comments 0 reactions 0 assignees View on GitHub
extensions java new feature P3 sketching
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 5h
Merged PRs (30d)
204

Description

Goal : Provide an extension library to compute approximate statistics on streams.

Interest : Probabilistic data structures can create an approximation (sketch) of the current state of a stream without storing every element but rather processing each observation quickly to summarize its current state and find useful statistical insights.

Implementation is here : https://github.com/ArnaudFnr/beam/tree/sketching/sdks/java/extensions/sketching

More info : https://docs.google.com/document/d/1Xy6g5RPBYX_HadpIr_2WrUeusiwL0Jo2ACI5PEOP1kc/edit

Imported from Jira [BEAM-2728](https://issues.apache.org/jira/browse/BEAM-2728). Original Jira may contain additional context.
Reported by: arnaudfnr.

Contributor guide

Open the contributing guide

Research direction

Review the proposed implementation in sdks/java/extensions/sketching and the linked design document first. Determine the extension’s intended approximate stream statistics and integration requirements from those materials. Done means the sketching extension is implemented and provides the described approximate statistics for streams.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.