Transform names and named applications should not be null or empty
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 204
Description
Beam SDK allows setting the name of a transform [1] and also naming the transform application [2]. If no name is specified on application, the name of the transform is used. If no name is specified for the transform, the class name is used.
The application name serves as metadata for the applied PTransforms in the constructed graph. The are effectively extra display data (historically, PTransform names predate display data). The names are used by runners for UI and monitoring applications, such as the displayed pipeline graph in the Dataflow Monitoring UI [3].
Currently there is no explicit validation on the specified application name. The current behavior seems to be:
* null application names cause a NullPointerException at construction time.
* Specifying the empty string compiles and succeeds in the DirectRunner, but causes strange behavior in Dataflow when rendering the graph in the UI. I have not tested the behavior of other runners.
We should add explicit validation in the model on the specified transform name and application name. I propose that we disallow null and empty names.
This is technically a breaking change as the SDK currently allows the empty string, but only because it is under-specified. The upgrade path for any pipelines broken by this change is simple: specify a non-empty name or fallback to the default class name.
[1] https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/PTransform.java#L236
[2] https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/values/PCollection.java#L295
[3] https://cloud.google.com/dataflow/pipelines/dataflow-monitoring-intf#viewing-a-pipeline
Imported from Jira [BEAM-2450](https://issues.apache.org/jira/browse/BEAM-2450). Original Jira may contain additional context.
Reported by: swegner.
Contributor guide
Research direction
Start with the name-handling code in sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/PTransform.java and sdks/java/core/src/main/java/org/apache/beam/sdk/values/PCollection.java, following the linked references. Check how transform and application names are constructed, then verify that null and empty specified names are explicitly rejected while the documented fallbacks remain available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend-api-design
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100