apache / apache/beam

[Bug] Python Mongodb IO error when loading collection with single document

Open
#23,090 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

**What happened?**

When I tried to read from a mongodb collection with only a single document in it, the worker returns "float division by zero" error. The source database is deployed in mongodb atlas and the pipeline is deployed on gcp dataflow runner. Below is the code we used and the error message got:

**Code**:

`pipeline | ReadFromMongoDB(uri=..., db=..., coll=..., bucket_auto=True)`

**ERROR**:
> Error message from worker: Traceback (most recent call last):
> File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line 646, in do_work
> work_executor.execute()
> File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py", line 208, in execute
> op.start()
> File "dataflow_worker/native_operations.py", line 38, in dataflow_worker.native_operations.NativeReadOperation.start
> File "dataflow_worker/native_operations.py", line 40, in dataflow_worker.native_operations.NativeReadOperation.start
> File "dataflow_worker/native_operations.py", line 44, in dataflow_worker.native_operations.NativeReadOperation.start
> File "dataflow_worker/native_operations.py", line 49, in dataflow_worker.native_operations.NativeReadOperation.start
> File "/usr/local/lib/python3.7/site-packages/dataflow_worker/workercustomsources.py", line 112, in get_progress
> fraction = self._range_tracker.fraction_consumed()
> File "/usr/local/lib/python3.7/site-packages/apache_beam/io/range_trackers.py", line 286, in fraction_consumed
> self._last_claim, self._start_position, self._stop_position)
> File "/usr/local/lib/python3.7/site-packages/apache_beam/io/range_trackers.py", line 432, in position_to_fraction
> return float(ikey - istart) / (iend - istart)
> ZeroDivisionError: float division by zero

Contributor guide

Open the contributing guide

Research direction

Start with the ReadFromMongoDB call and the traceback in apache_beam/io/range_trackers.py, especially position_to_fraction and fraction_consumed. Reproduce the pipeline with a MongoDB collection containing one document, then verify that the document loads without the reported ZeroDivisionError.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, mongodb, python
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.