apache / apache/hop

[Task]: add context about using Group By in distributed environments (Beam)

Open
#3,360 1 comment 0 reactions 0 assignees View on GitHub
awaiting triage Beam Documentation P2 Transforms
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 32m
Merged PRs (30d)
216

Description

### What needs to happen?

the Group By requires sorted input. In a distributed environment (like the Beam run configurations), sorting happens per node, which may lead to unexpected results.
This should be made more clear in the docs, we should point users to the Memory Group By transform, which doesn't require sorted input.

### Issue Priority

Priority: 2

### Issue Component

Component: Beam, Component: Documentation, Component: Transforms

Contributor guide

Open the contributing guide

Research direction

Locate the documentation for the Group By and Memory Group By transforms, then review how Beam run configurations are described. Clarify that Group By requires sorted input and that distributed sorting is per node, and point readers to Memory Group By as the alternative; done when the limitation and alternative are clear in the relevant docs.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems, documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.