apache / apache/hop

[Feature Request]: Apache Beam partitioning support

Open
#2,040 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
19h
Merged PRs (30d)
227

Description

### What would you like to happen?

Implement partitioning support, present in the local Hop pipeline engine, for Beam pipelines.

https://beam.apache.org/documentation/transforms/java/elementwise/partition/

### Issue Priority

Priority: 3

### Issue Component

Component: Beam

Contributor guide

Open the contributing guide

Research direction

Start by comparing partitioning in the local Hop pipeline engine with the Apache Beam Partition transform documentation linked in the issue. Identify the Beam pipeline entry points and existing tests for similar transforms; done means Beam pipelines support the requested partitioning behavior consistently with the local engine.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.