apache / apache/hop

[Bug]: Pipeline action - record set forwarding issue

Open
#4,559 0 comments 0 reactions 0 assignees View on GitHub
Actions awaiting triage P3 Pipelines Transforms Workflows
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 32m
Merged PRs (30d)
216

Description

### Apache Hop version?

2.10.0

### Java version?

17.0.2

### Operating system

Linux

### What happened?

Steps to reproduce:
1. Create two pipelines, the first ending with a Filter rows and a Copy rows to result, the second starting with a Get rows from result
2. Link those pipelines together in a workflow

If the Filter rows transform gives some rows to the next Copy rows to result, the filtered record set is correctly forwarded to the next pipeline.
If the Filter rows doesn't give rows in output, I'd expect the next pipeline to be skipped: instead, it will be processing a record set anyway (I need to check if it's forwarding data from an earlier pipeline, or if it's getting data from another transform in the first pipeline)

### Issue Priority

Priority: 3

### Issue Component

Component: Actions, Component: Pipelines, Component: Transforms, Component: Workflows

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with two pipelines: Filter rows followed by Copy rows to result, then Get rows from result in the next pipeline within a workflow. Trace the action and pipeline handoff when the filter produces no output, and verify that the next pipeline receives no record set.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.