apache / apache/hop

[Bug]: Join rows (cartesian product) hangs on Beam

Open
#2,353 1 comment 0 reactions 0 assignees View on GitHub
awaiting triage Beam P3
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 33m
Merged PRs (30d)
204

Description

### Apache Hop version?

2.3

### Java version?

11

### Operating system

Windows

### What happened?

Migration: https://issues.apache.org/jira/browse/HOP-4185

A pipeline containing a "Join rows (cartesian product)" transform with input from two sources loops forever when run via Beam-Direct or Dataflow. Attaching a debugger shows that it is stuck in the while loop at [JoinRows.java:486](https://github.com/apache/hop/blob/758c07c360c26c0447251f0a29df81557864ad11/plugins/transforms/joinrows/src/main/java/org/apache/hop/pipeline/transforms/joinrows/JoinRows.java#L487).

### Issue Priority

Priority: 3

### Issue Component

Component: Beam

Contributor guide

Open the contributing guide

Research direction

Start with plugins/transforms/joinrows/src/main/java/org/apache/hop/pipeline/transforms/joinrows/JoinRows.java around line 486, and reproduce the pipeline with two input sources through Beam-Direct or Dataflow. Trace the while loop with the debugger and compare its behavior with the non-Beam execution. Done means the Join rows (cartesian product) transform completes instead of looping forever on Beam.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.