apache / apache/beam

Factor execution time and construction time dependencies into a non-SDK component

Open
#18,572 0 comments 0 reactions 0 assignees View on GitHub
core java new feature P3 runners
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

Generally an execution engine should not depend on SDK constructs (e.g. DoFn), as those should be used by the language's SDK harness to execute a UDF. However, there are certain constructs that are used to communicate between the Runner and SDK harness, or are required to fulfill the Runner's responsibilities (e.g. Coder, Window). Those should be accessible without having to depend on the entire SDK

Imported from Jira [BEAM-3100](https://issues.apache.org/jira/browse/BEAM-3100). Original Jira may contain additional context.
Reported by: tgroh.

Contributor guide

Open the contributing guide

Research direction

Start by reading the issue and its linked BEAM-3100 Jira entry to recover the missing context and identify the execution-time and construction-time dependencies. Map which Runner/SDK-harness communication constructs, especially Coder and Window, are required without the full SDK. Done means those constructs are accessible from a non-SDK component while execution engines no longer depend on SDK constructs such as DoFn.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.