[Bug]: Example sql_taxi.py not working with DirectRunner
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What happened?
I am trying to run the example [https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/sql_taxi.py](sdks/python/apache_beam/examples/sql_taxi.py) on my local machine. After some minor adaptations (I created a subscription to the "taxirides-realtime" topic and reference that instead of the topic itself), I expected it to work. But I get a NotImplementedError that indicates that SqlTransform is not implemented in DirectRunner.
The stacktrace I get:
```python
ERROR:apache_beam.runners.direct.executor:Exception at bundle , due to an exception.
Traceback (most recent call last):
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 370, in call
self.attempt_call(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 400, in attempt_call
evaluator = self._transform_evaluator_registry.get_evaluator(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 135, in get_evaluator
raise NotImplementedError(
NotImplementedError: Execution of [] not implemented in runner .
ERROR:apache_beam.runners.direct.executor:Exception at bundle , due to an exception.
Traceback (most recent call last):
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 370, in call
self.attempt_call(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 400, in attempt_call
evaluator = self._transform_evaluator_registry.get_evaluator(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 135, in get_evaluator
raise NotImplementedError(
NotImplementedError: Execution of [] not implemented in runner .
ERROR:apache_beam.runners.direct.executor:Exception at bundle , due to an exception.
Traceback (most recent call last):
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 370, in call
self.attempt_call(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 400, in attempt_call
evaluator = self._transform_evaluator_registry.get_evaluator(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 135, in get_evaluator
raise NotImplementedError(
NotImplementedError: Execution of [] not implemented in runner .
ERROR:apache_beam.runners.direct.executor:Exception at bundle , due to an exception.
Traceback (most recent call last):
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 370, in call
self.attempt_call(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 400, in attempt_call
evaluator = self._transform_evaluator_registry.get_evaluator(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 135, in get_evaluator
raise NotImplementedError(
NotImplementedError: Execution of [] not implemented in runner .
ERROR:apache_beam.runners.direct.executor:Giving up after 4 attempts.
WARNING:apache_beam.runners.direct.executor:A task failed with exception: Execution of [] not implemented in runner .
Traceback (most recent call last):
File "/Users/ture.friese/atl-dataflow-sql/sql_taxi.py", line 97, in
run(known_args.output_topic, pipeline_args)
File "/Users/ture.friese/atl-dataflow-sql/sql_taxi.py", line 46, in run
with beam.Pipeline(options=pipeline_options) as pipeline:
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/pipeline.py", line 598, in __exit__
self.result.wait_until_finish()
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/direct_runner.py", line 588, in wait_until_finish
self._executor.await_completion()
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 432, in await_completion
self._executor.await_completion()
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 480, in await_completion
raise update.exception
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 370, in call
self.attempt_call(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/executor.py", line 400, in attempt_call
evaluator = self._transform_evaluator_registry.get_evaluator(
File "/Users/ture.friese/miniconda3/envs/atldfs/lib/python3.10/site-packages/apache_beam/runners/direct/transform_evaluator.py", line 135, in get_evaluator
raise NotImplementedError(
NotImplementedError: Execution of [] not implemented in runner .
```
### Issue Priority
Priority: 2
### Issue Component
Component: examples-python
Contributor guide
Assessment
This issue has not been assessed yet.