apache / apache/hop

[Feature Request]: Add "Pass Input Fields to Output" Option in Mapping/Pipeline Executor Transforms

Open
#6,383 5 comments 0 reactions 0 assignees View on GitHub
awaiting triage P3 Pipelines
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
19h
Merged PRs (30d)
227

Description

### What would you like to happen?

Description: Issue Type: Improvement / Feature Request Priority: Major

Summary: Currently, when using a Mapping (Simple Mapping) or Pipeline Executor transform, fields from the main pipeline's input stream are not automatically propagated to the output step unless they are explicitly defined in the "Mapping Input" within the sub-pipeline and mapped back out.

If a developer simply wants to "enrich" a stream using a sub-pipeline (mapping) but keep all original fields (IDs, metadata, etc.), they are forced to manually map every single field inside the sub-pipeline configuration. This is time-consuming and prone to errors if the upstream data structure changes.

Proposed Solution: Add a checkbox or toggle option in the configuration dialog (e.g., "Main" or "Parameters" tab) labeled: "Pass all input fields to output" or "Propagate parent fields".

Behavior:

Enabled: All fields entering the Mapping transform are automatically available in the output stream, merged with any new fields created by the mapping.

Disabled: (Current behavior) Only fields explicitly returned by the Mapping Output are passed to the next step.

Benefit: This would significantly reduce development time for reusable components where the sub-pipeline performs a specific calculation or lookup but needs to preserve the original row context (IDs, timestamps) without manual mapping overhead.

### Issue Priority

Priority: 3

### Issue Component

Component: Pipelines

Contributor guide

Open the contributing guide

Research direction

Start by locating the Mapping (Simple Mapping) and Pipeline Executor transform configuration dialogs, especially their Main or Parameters tabs, then trace how input and output fields are handled. Review existing transform tests if present; done means an enabled option preserves parent input fields alongside mapped fields, while disabled behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.