[Bug] Python Mongodb IO error when loading collection with single document
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
**What happened?**
When I tried to read from a mongodb collection with only a single document in it, the worker returns "float division by zero" error. The source database is deployed in mongodb atlas and the pipeline is deployed on gcp dataflow runner. Below is the code we used and the error message got:
**Code**:
`pipeline | ReadFromMongoDB(uri=..., db=..., coll=..., bucket_auto=True)`
**ERROR**:
> Error message from worker: Traceback (most recent call last):
> File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line 646, in do_work
> work_executor.execute()
> File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py", line 208, in execute
> op.start()
> File "dataflow_worker/native_operations.py", line 38, in dataflow_worker.native_operations.NativeReadOperation.start
> File "dataflow_worker/native_operations.py", line 40, in dataflow_worker.native_operations.NativeReadOperation.start
> File "dataflow_worker/native_operations.py", line 44, in dataflow_worker.native_operations.NativeReadOperation.start
> File "dataflow_worker/native_operations.py", line 49, in dataflow_worker.native_operations.NativeReadOperation.start
> File "/usr/local/lib/python3.7/site-packages/dataflow_worker/workercustomsources.py", line 112, in get_progress
> fraction = self._range_tracker.fraction_consumed()
> File "/usr/local/lib/python3.7/site-packages/apache_beam/io/range_trackers.py", line 286, in fraction_consumed
> self._last_claim, self._start_position, self._stop_position)
> File "/usr/local/lib/python3.7/site-packages/apache_beam/io/range_trackers.py", line 432, in position_to_fraction
> return float(ikey - istart) / (iend - istart)
> ZeroDivisionError: float division by zero
Contributor guide
Research direction
Start with the ReadFromMongoDB call and the traceback in apache_beam/io/range_trackers.py, especially position_to_fraction and fraction_consumed. Reproduce the pipeline with a MongoDB collection containing one document, then verify that the document loads without the reported ZeroDivisionError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, mongodb, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100