apache / apache/beam

Division By Zero errors being thrown by new BQ Source

Open
#20,789 0 comments 0 reactions 0 assignees View on GitHub
dataflow gcp improvement io P3 python runners
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Users are reporting issues with the BQ Source, where division by zero errors are being reported by Beam. A stack trace:
```

Log: [1] An exception was raised when trying to execute the workitem 2053702133423558223 : Traceback
(most recent call last): File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py",
line 649, in do_work work_executor.execute() File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py",
line 179, in execute op.start() File "dataflow_worker/native_operations.py", line 38, in dataflow_worker.native_operations.NativeReadOperation.start
File "dataflow_worker/native_operations.py", line 39, in dataflow_worker.native_operations.NativeReadOperation.start
File "dataflow_worker/native_operations.py", line 44, in dataflow_worker.native_operations.NativeReadOperation.start
File "dataflow_worker/native_operations.py", line 48, in dataflow_worker.native_operations.NativeReadOperation.start
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/workercustomsources.py", line 69, in iter self._source.start_position,
self._source.stop_position) File "/usr/local/lib/python3.7/site-packages/apache_beam/io/concat_source.py",
line 78, in get_range_tracker start_position, stop_position, self._source_bundles) File "/usr/local/lib/python3.7/site-packages/apache_beam/io/concat_source.py",
line 131, in init self._compute_cumulative_weights(source_bundles[start[0]:last]) + [1] * File "/usr/local/lib/python3.7/site-packages/apache_beam/io/concat_source.py",
line 154, in _compute_cumulative_weights running_total.append(max(min_diff, min(1, running_total[-1]
+ w / total))) ZeroDivisionError: flo

```

 

I am suspecting this issue: https://issues.apache.org/jira/browse/BEAM-2716
[](https://issues.apache.org/jira/secure/AddComment!default.jspa?id=13341887)

Imported from Jira [BEAM-11497](https://issues.apache.org/jira/browse/BEAM-11497). Original Jira may contain additional context.
Reported by: pabloem.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.