apache / apache/beam

PBegin | Write produces obscure error

Open
#18,237 0 comments 0 reactions 0 assignees View on GitHub
core improvement P3 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Repro: {{p| beam.io.WriteToText(output_file))||

produces:
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/pipeline.py", line 157, in run
self.to_runner_api(), self.runner, self.options).run(False)
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/pipeline.py", line 334, in to_runner_api
root_transform_id = context.transforms.get_id(self._root_transform())
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/runners/pipeline_context.py", line 52, in get_id
self._id_to_proto[id] = obj.to_runner_api(self._pipeline_context)
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/pipeline.py", line 523, in to_runner_api
subtransforms=[context.transforms.get_id(part) for part in self.parts],
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/runners/pipeline_context.py", line 52, in get_id
self._id_to_proto[id] = obj.to_runner_api(self._pipeline_context)
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/pipeline.py", line 523, in to_runner_api
subtransforms=[context.transforms.get_id(part) for part in self.parts],
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/runners/pipeline_context.py", line 52, in get_id
self._id_to_proto[id] = obj.to_runner_api(self._pipeline_context)
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/pipeline.py", line 528, in to_runner_api
for tag, out in self.named_outputs().items()},
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/pipeline.py", line 528, in
for tag, out in self.named_outputs().items()},
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/runners/pipeline_context.py", line 52, in get_id
self._id_to_proto[id] = obj.to_runner_api(self._pipeline_context)
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/pvalue.py", line 128, in to_runner_api
self.windowing))
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/pvalue.py", line 110, in windowing
self.producer.inputs)
File "/Users/tiborkiss/workspace/beam/sdks/python/apache_beam/transforms/ptransform.py", line 336, in get_windowing
return inputs[0].windowing
AttributeError: 'PBegin' object has no attribute 'windowing'

cc: [~tibor.kiss@gmail.com]

Imported from Jira [BEAM-2107](https://issues.apache.org/jira/browse/BEAM-2107). Original Jira may contain additional context.
Reported by: altay.

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with beam.io.WriteToText and trace the reported path through sdks/python/apache_beam/pipeline.py, pvalue.py, and transforms/ptransform.py. Determine how this PBegin case should be reported; done means the same invalid usage produces an understandable error instead of the shown AttributeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.