apache / apache/hop

[Bug]: Setting a variable to a scope of 'valid in the JVM' does not work upstream a repeating pipeline in a workflow

Open
#3,630 0 comments 0 reactions 0 assignees View on GitHub
awaiting triage P3 Pipelines Workflows
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 33m
Merged PRs (30d)
204

Description

### Apache Hop version?

2.8.0-SNAPSHOT (2024-02-01 04.40.17)

### Java version?

openjdk version "11.0.19" 2023-04-18 LTS OpenJDK Runtime Environment Microsoft-7621296 (build 11.0.19+7-LTS) OpenJDK 64-Bit Server VM Microsoft-7621296 (build 11.0.19+7-LTS, mixed mode)

### Operating system

Windows

### What happened?

NOTE that scope of 'valid in JVM' works without a repeating pipeline and if top level is set to "the output will contain result rows after execution". If you use a repeating pipeline it will not work ...
[Pipeline_to_workflow_to_pipeline_all_the_way_up.zip](https://github.com/apache/hop/files/14301111/Pipeline_to_workflow_to_pipeline_all_the_way_up.zip)

THE TEST:
top level: Pipeline: call workflow
mid level: Workflow: call repeating pipeline
low level: Pipeline: set/send variable all the way back to top level
Set variable so as to read it in the top-most level (pipeline)

RESULTS from setting variable in lowest pipeline:
1) set scope to valid in root or parent workflow: A) PASS to mid level, B) FAIL to top level.
2) set scope to valid in JVM or grand parent workflow: FAIL at mid level and top level

Please see 3 test files attached.

### Issue Priority

Priority: 3

### Issue Component

Component: Pipelines, Component: Workflows

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the attached Pipeline_to_workflow_to_pipeline_all_the_way_up.zip scenario with the three attached test files: a top-level pipeline, a workflow calling a repeating pipeline, and the lowest pipeline setting the variable. Trace variable propagation for the reported scopes and compare behavior with and without repetition. Done means the JVM or grand-parent scope, and the root or parent workflow scope, reach the top-level pipeline consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.