apache / apache/beam

Transform names and named applications should not be null or empty

Open
#18,465 0 comments 0 reactions 0 assignees View on GitHub
beam-model bug core java P3 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 5h
Merged PRs (30d)
204

Description

Beam SDK allows setting the name of a transform [1] and also naming the transform application [2]. If no name is specified on application, the name of the transform is used. If no name is specified for the transform, the class name is used.

The application name serves as metadata for the applied PTransforms in the constructed graph. The are effectively extra display data (historically, PTransform names predate display data). The names are used by runners for UI and monitoring applications, such as the displayed pipeline graph in the Dataflow Monitoring UI [3].

Currently there is no explicit validation on the specified application name. The current behavior seems to be:
* null application names cause a NullPointerException at construction time.
* Specifying the empty string compiles and succeeds in the DirectRunner, but causes strange behavior in Dataflow when rendering the graph in the UI. I have not tested the behavior of other runners.

We should add explicit validation in the model on the specified transform name and application name. I propose that we disallow null and empty names.

This is technically a breaking change as the SDK currently allows the empty string, but only because it is under-specified. The upgrade path for any pipelines broken by this change is simple: specify a non-empty name or fallback to the default class name.

[1] https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/PTransform.java#L236
[2] https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/values/PCollection.java#L295
[3] https://cloud.google.com/dataflow/pipelines/dataflow-monitoring-intf#viewing-a-pipeline

Imported from Jira [BEAM-2450](https://issues.apache.org/jira/browse/BEAM-2450). Original Jira may contain additional context.
Reported by: swegner.

Contributor guide

Open the contributing guide

Research direction

Start with the name-handling code in sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/PTransform.java and sdks/java/core/src/main/java/org/apache/beam/sdk/values/PCollection.java, following the linked references. Check how transform and application names are constructed, then verify that null and empty specified names are explicitly rejected while the documented fallbacks remain available.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.