apache / apache/hop

[Bug]: I use hop to run Pipeline in flink, can I get the JobID of Flink?

Open
#3,131 1 comment 0 reactions 0 assignees View on GitHub
awaiting triage Beam P3
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 33m
Merged PRs (30d)
204

Description

### Apache Hop version?

2.4

### Java version?

11

### Operating system

Windows

### What happened?

I used the following to create IPipelineEngine
`IPipelineEngine pipeline = PipelineEngineFactory.createPipelineEngine(
variables,
executionConfiguration.getRunConfiguration(),
pipelineMeta.getMetadataProvider(),
pipelineMeta);`

Then start executing

` pipeline.prepareExecution();
pipeline.startThreads();
`
I will submit the Pipeline to the Flink environment to run, so can I get the JobID of this Pipeline in Flink through code?
Something like
![image](https://github.com/apache/hop/assets/58242269/280effd2-0ff5-4436-b054-ee4424ed6af3)

### Issue Priority

Priority: 3

### Issue Component

Component: Beam

Contributor guide

Open the contributing guide

Research direction

Start with the IPipelineEngine creation and execution calls shown in the issue, then inspect the Beam/Flink run configuration and execution entry points to determine whether the Flink JobID is exposed. Done means the issue has a documented or implemented way to retrieve the submitted Pipeline's JobID through code.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.