apache / apache/beam

Python SDK does not correctly set the serialized_fn of WindowInto when translating to the Dataflow pipeline description

Open
#19,034 0 comments 0 reactions 0 assignees View on GitHub
bug dataflow P3 portability runners
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

serialzed_fn at the bottom should be reference to the pipeline proto ptransform id representing this function.
```

{
"kind": "ParallelDo",
"name": "s4",
"properties": {
"display_data":
[
{
"key": "fn",
"label": "Transform Function",

"namespace": "apache_beam.transforms.core.WindowInto",
"shortValue": "WindowIntoFn",

"type": "STRING",
"value": "apache_beam.transforms.core.WindowIntoFn"

}
],
"non_parallel_inputs": {},
"output_info": [
{

"encoding": {
"@type": "kind:windowed_value",
"component_encodings":
[
{
"@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",

"component_encodings": [
{
"@type":
"FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",

"component_encodings": []
},
{

"@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",

"component_encodings": []
}
],

"is_pair_like": true
},
{
"@type":
"kind:global_window"
}
],
"is_wrapper": true

},
"output_name": "out",
"user_name": "assert_that/WindowInto(WindowIntoFn).out"

}
],
"parallel_input": {
"@type": "OutputReference",

"output_name": "out",
"step_name": "s2"
},
"serialized_fn": "",
"user_name": "assert_that/WindowInto(WindowIntoFn)"
}
},

```

Imported from Jira [BEAM-4581](https://issues.apache.org/jira/browse/BEAM-4581). Original Jira may contain additional context.
Reported by: lcwik.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.