GoogleCloudPlatform / GoogleCloudPlatform/training-data-analyst

AttributeError: Can't get attribute '_create_code' on <module 'dill in feateng.ipydb

Open
#1,545 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
8.6k
Forks
6.1k
Avg merge
4h 44m
Merged PRs (30d)
2

Description

Trying to execute step 4 "Run Beam pipeline on Cloud Dataflow" in `training-data-analyst/courses/machine_learning/feateng/feateng.ipynb`, it consistently fails with the following error:

```
/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:54: BeamDeprecationWarning: BigQuerySource is deprecated since 2.25.0. Use ReadFromBigQuery instead.
/opt/conda/lib/python3.7/site-packages/apache_beam/io/gcp/bigquery.py:1927: BeamDeprecationWarning: options is deprecated since First stable release. References to .options will not be supported
temp_location = pcoll.pipeline.options.view_as(
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.7 interpreter.
---------------------------------------------------------------------------
DataflowRuntimeException Traceback (most recent call last)
in
----> 1 preprocess(50*100, 'DataflowRunner')

in preprocess(EVERY_N, RUNNER)
52 p | 'read_{}'.format(phase) >> beam.io.Read(beam.io.BigQuerySource(query=query))
53 | 'tocsv_{}'.format(phase) >> beam.Map(to_csv)
---> 54 | 'write_{}'.format(phase) >> beam.io.Write(beam.io.WriteToText(outfile))
55 )
56 print("Done")

/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py in __exit__(self, exc_type, exc_val, exc_tb)
584 if not exc_type:
585 self.result = self.run()
--> 586 self.result.wait_until_finish()
587 finally:
588 self._extra_context.__exit__(exc_type, exc_val, exc_tb)

/opt/conda/lib/python3.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py in wait_until_finish(self, duration)
1627 'Dataflow pipeline failed. State: %s, Error:\n%s' %
1628 (self.state, getattr(self._runner, 'last_error_msg', None)),
-> 1629 self)
1630 return self.state
1631

DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py", line 283, in loads
return dill.loads(s)
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 275, in loads
return load(file, ignore, **kwds)
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 270, in load
return Unpickler(file, ignore=ignore, **kwds).load()
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 472, in load
obj = StockUnpickler.load(self)
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 462, in find_class
return StockUnpickler.find_class(self, module, name)
AttributeError: Can't get attribute '_create_code' on

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line 651, in do_work
work_executor.execute()
File "/usr/local/lib/python3.7/site-packages/dataflow_worker/executor.py", line 179, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 709, in apache_beam.runners.worker.operations.DoOperation.start
File "apache_beam/runners/worker/operations.py", line 711, in apache_beam.runners.worker.operations.DoOperation.start
File "apache_beam/runners/worker/operations.py", line 712, in apache_beam.runners.worker.operations.DoOperation.start
File "apache_beam/runners/worker/operations.py", line 310, in apache_beam.runners.worker.operations.Operation.start
File "apache_beam/runners/worker/operations.py", line 316, in apache_beam.runners.worker.operations.Operation.start
File "apache_beam/runners/worker/operations.py", line 658, in apache_beam.runners.worker.operations.DoOperation.setup
File "apache_beam/runners/worker/operations.py", line 659, in apache_beam.runners.worker.operations.DoOperation.setup
File "apache_beam/runners/worker/operations.py", line 291, in apache_beam.runners.worker.operations.Operation.setup
File "apache_beam/runners/worker/operations.py", line 305, in apache_beam.runners.worker.operations.Operation.setup
File "apache_beam/runners/worker/operations.py", line 798, in apache_beam.runners.worker.operations.DoOperation._get_runtime_performance_hints
File "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py", line 287, in loads
return dill.loads(s)
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 275, in loads
return load(file, ignore, **kwds)
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 270, in load
return Unpickler(file, ignore=ignore, **kwds).load()
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 472, in load
obj = StockUnpickler.load(self)
File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 462, in find_class
return StockUnpickler.find_class(self, module, name)
AttributeError: Can't get attribute '_create_code' on
```

Random Googling indicates that this is due to a Python version incompatibility.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.