apache / apache/beam

Splittable DoFn and Dataflow, "conflicting bucketing functions"

Open
#20,543 0 comments 0 reactions 0 assignees View on GitHub
bug dataflow P3 runners
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

When creating a pipeline with an SDF like:
FileIO.match -\> readMatches -\> drive the upstream side input
OR
[unbounded watcher, sdf] -\> [ParDo with side input from File.IO] -\> [bounded sdf] -\> [ParDo]

Workflow failed. Causes: Step s22 has conflicting bucketing functions

Attached to this bug is a repro supplied by the user (TsPointsStreamingFail.java)

Source for reported issue:
https://lists.apache.org/thread.html/r03c77ea03d7ff2678052bde412da19c4e13050652fd34d3e03a9e30f%40%3Cuser.beam.apache.org%3E

Imported from Jira [BEAM-10914](https://issues.apache.org/jira/browse/BEAM-10914). Original Jira may contain additional context.
Reported by: lcwik.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.