GoogleCloudPlatform / GoogleCloudPlatform/training-data-analyst

"Collaborative Filtering on Google Analytics data" lab fails with `NotImplementedError: Cannot convert a symbolic Tensor (cond_1/strided_slice_4:0) to a numpy array.`

Open
#2,441 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
8.6k
Forks
6.1k
Avg merge
4h 44m
Merged PRs (30d)
2

Description

Just like the lab in #2439 this one (https://www.cloudskillsboost.google/course_sessions/2920313/labs/325084) is also part of the recommendation course (of the Professional Machine Learning Engineer Path), and also uses 1.15 TF. It set off on the right foot by using a CLI command to instantiate the Vertex AI Workbench Anaconda VM, since 1.15 TF is not selectable on the User Managed Notebook creation GUI any more.

The notebook spews warnings left and right (which we can usually ignore), but it failed at the `Run as a Python module` step's `gcloud ai-platform local train` CLI step. The root cause looked to be `NotImplementedError: Cannot convert a symbolic Tensor (cond_1/strided_slice_4:0) to a numpy array.`

```
WARNING:tensorflow:From /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py:27: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py:27: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

WARNING:tensorflow:From /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py:198: run (from tensorflow.contrib.learn.python.learn.learn_runner) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.estimator.train_and_evaluate.
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
* https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
* https://github.com/tensorflow/addons
* https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/estimators/estimator.py:1180: BaseEstimator.__init__ (from tensorflow.contrib.learn.python.learn.estimators.estimator) is deprecated and will be removed in a future version.
Instructions for updating:
Please replace uses of any Estimator from tf.contrib.learn with an Estimator from tf.estimator.*
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/estimators/estimator.py:427: RunConfig.__init__ (from tensorflow.contrib.learn.python.learn.estimators.run_config) is deprecated and will be removed in a future version.
Instructions for updating:
When switching to tf.estimator.Estimator, use tf.estimator.RunConfig instead.
INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_task_type': None, '_task_id': 0, '_cluster_spec': , '_master': '', '_num_ps_replicas': 0, '_num_worker_replicas': 0, '_environment': 'cloud', '_is_chief': True, '_evaluation_master': '', '_train_distribute': None, '_eval_distribute': None, '_experimental_max_worker_delay_secs': None, '_device_fn': None, '_tf_config': gpu_options {
per_process_gpu_memory_fraction: 1.0
}
, '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_secs': 600, '_log_step_count_steps': 100, '_protocol': None, '_session_config': None, '_save_checkpoints_steps': None, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_model_dir': '/home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/wals_trained/', '_session_creation_timeout_secs': 7200}
WARNING:tensorflow:From /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py:194: make_export_strategy (from tensorflow.contrib.learn.python.learn.utils.saved_model_export_utils) is deprecated and will be removed in a future version.
Instructions for updating:
Switch to tf.estimator.Exporter and associated utilities.
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/utils/saved_model_export_utils.py:484: ExportStrategy.__new__ (from tensorflow.contrib.learn.python.learn.export_strategy) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.estimator.train_and_evaluate, and use tf.estimator.Exporter.
WARNING:tensorflow:From /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py:194: Experiment.__init__ (from tensorflow.contrib.learn.python.learn.experiment) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.estimator.train_and_evaluate. You will also have to convert to a tf.estimator.Estimator.
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/monitors.py:279: BaseMonitor.__init__ (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.
Instructions for updating:
Monitors are deprecated. Please use tf.train.SessionRunHook.
WARNING:tensorflow:From /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py:85: The name tf.gfile.Glob is deprecated. Please use tf.io.gfile.glob instead.

WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.

WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.VarLenFeature is deprecated. Please use tf.io.VarLenFeature instead.

WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.parse_single_example is deprecated. Please use tf.io.parse_single_example instead.

WARNING:tensorflow:From /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py:41: sparse_merge (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
No similar op available at this time.
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py:1475: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py:93: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `for ... in dataset:` to iterate over a dataset. If using `tf.estimator`, return the `Dataset` object directly from your input function. As a last resort, you can use `tf.compat.v1.data.make_one_shot_iterator(dataset)`.
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/factorization/python/ops/wals.py:315: ModelFnOps.__new__ (from tensorflow.contrib.learn.python.learn.estimators.model_fn) is deprecated and will be removed in a future version.
Instructions for updating:
When switching to tf.estimator.Estimator, use tf.estimator.EstimatorSpec. You can use the `estimator_spec` method to create an equivalent one.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.

User settings:

KMP_AFFINITY=granularity=fine,verbose,compact,1,0
KMP_BLOCKTIME=0
KMP_DUPLICATE_LIB_OK=True
KMP_INIT_AT_FORK=FALSE
KMP_SETTINGS=1
OMP_NUM_THREADS=1

Effective settings:

KMP_ABORT_DELAY=0
KMP_ADAPTIVE_LOCK_PROPS='1,1024'
KMP_ALIGN_ALLOC=64
KMP_ALL_THREADPRIVATE=128
KMP_ATOMIC_MODE=2
KMP_BLOCKTIME=0
KMP_CPUINFO_FILE: value is not defined
KMP_DETERMINISTIC_REDUCTION=false
KMP_DEVICE_THREAD_LIMIT=2147483647
KMP_DISP_HAND_THREAD=false
KMP_DISP_NUM_BUFFERS=7
KMP_DUPLICATE_LIB_OK=true
KMP_FORCE_REDUCTION: value is not defined
KMP_FOREIGN_THREADS_THREADPRIVATE=true
KMP_FORKJOIN_BARRIER='2,2'
KMP_FORKJOIN_BARRIER_PATTERN='hyper,hyper'
KMP_FORKJOIN_FRAMES=true
KMP_FORKJOIN_FRAMES_MODE=3
KMP_GTID_MODE=3
KMP_HANDLE_SIGNALS=false
KMP_HOT_TEAMS_MAX_LEVEL=1
KMP_HOT_TEAMS_MODE=0
KMP_INIT_AT_FORK=true
KMP_ITT_PREPARE_DELAY=0
KMP_LIBRARY=throughput
KMP_LOCK_KIND=queuing
KMP_MALLOC_POOL_INCR=1M
KMP_MWAIT_HINTS=0
KMP_NUM_LOCKS_IN_BLOCK=1
KMP_PLAIN_BARRIER='2,2'
KMP_PLAIN_BARRIER_PATTERN='hyper,hyper'
KMP_REDUCTION_BARRIER='1,1'
KMP_REDUCTION_BARRIER_PATTERN='hyper,hyper'
KMP_SCHEDULE='static,balanced;guided,iterative'
KMP_SETTINGS=true
KMP_SPIN_BACKOFF_PARAMS='4096,100'
KMP_STACKOFFSET=64
KMP_STACKPAD=0
KMP_STACKSIZE=8M
KMP_STORAGE_MAP=false
KMP_TASKING=2
KMP_TASKLOOP_MIN_TASKS=0
KMP_TASK_STEALING_CONSTRAINT=1
KMP_TEAMS_THREAD_LIMIT=1
KMP_TOPOLOGY_METHOD=all
KMP_USER_LEVEL_MWAIT=false
KMP_USE_YIELD=1
KMP_VERSION=false
KMP_WARNINGS=true
OMP_AFFINITY_FORMAT='OMP: pid %P tid %i thread %n bound to OS proc set {%A}'
OMP_ALLOCATOR=omp_default_mem_alloc
OMP_CANCELLATION=false
OMP_DEBUG=disabled
OMP_DEFAULT_DEVICE=0
OMP_DISPLAY_AFFINITY=false
OMP_DISPLAY_ENV=false
OMP_DYNAMIC=false
OMP_MAX_ACTIVE_LEVELS=2147483647
OMP_MAX_TASK_PRIORITY=0
OMP_NESTED=false
OMP_NUM_THREADS='1'
OMP_PLACES: value is not defined
OMP_PROC_BIND='intel'
OMP_SCHEDULE='static'
OMP_STACKSIZE=8M
OMP_TARGET_OFFLOAD=DEFAULT
OMP_THREAD_LIMIT=2147483647
OMP_TOOL=enabled
OMP_TOOL_LIBRARIES: value is not defined
OMP_WAIT_POLICY=PASSIVE
KMP_AFFINITY='verbose,warnings,respect,granularity=fine,compact,1,0'

2023-09-13 19:24:41.488800: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2299995000 Hz
2023-09-13 19:24:41.488984: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x556f96f35350 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2023-09-13 19:24:41.489018: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2023-09-13 19:24:41.489118: I tensorflow/core/common_runtime/process_util.cc:136] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/wals_trained/model.ckpt.
INFO:tensorflow:SweepHook running init op.
INFO:tensorflow:SweepHook running prep ops for the row sweep.
OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 0
OMP: Info #156: KMP_AFFINITY: 1 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology
OMP: Info #159: KMP_AFFINITY: 1 packages x 1 cores/pkg x 1 threads/core (1 total cores)
OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #171: KMP_AFFINITY: OS proc 0 maps to package 0
OMP: Info #250: KMP_AFFINITY: pid 2369 tid 2396 thread 0 bound to OS proc set 0
OMP: Info #250: KMP_AFFINITY: pid 2369 tid 2396 thread 1 bound to OS proc set 0
INFO:tensorflow:Next fit step starting.
OMP: Info #250: KMP_AFFINITY: pid 2369 tid 2398 thread 2 bound to OS proc set 0
OMP: Info #250: KMP_AFFINITY: pid 2369 tid 2395 thread 3 bound to OS proc set 0
INFO:tensorflow:loss = 171085.25, step = 1
INFO:tensorflow:Next fit step starting.
INFO:tensorflow:Saving checkpoints for 2 into /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/wals_trained/model.ckpt.
INFO:tensorflow:Loss for final step: 170297.7.
INFO:tensorflow:Starting evaluation at 2023-09-13T19:24:43Z
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/wals_trained/model.ckpt-2
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
OMP: Info #250: KMP_AFFINITY: pid 2369 tid 2440 thread 2 bound to OS proc set 0
INFO:tensorflow:Evaluation [1/1]
INFO:tensorflow:Finished evaluation at 2023-09-13-19:24:44
INFO:tensorflow:Saving dict for global step 2: global_step = 2, loss = 171085.25
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/estimators/estimator.py:1374: get_timestamped_export_dir (from tensorflow.contrib.learn.python.learn.utils.saved_model_export_utils) is deprecated and will be removed in a future version.
Instructions for updating:
Switch to tf.estimator.Exporter and associated utilities.
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/estimators/estimator.py:1379: get_temp_export_dir (from tensorflow.contrib.learn.python.learn.utils.saved_model_export_utils) is deprecated and will be removed in a future version.
Instructions for updating:
Switch to tf.estimator.Exporter and associated utilities.
WARNING:tensorflow:From /home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py:157: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/estimators/estimator.py:1389: get_input_alternatives (from tensorflow.contrib.learn.python.learn.utils.saved_model_export_utils) is deprecated and will be removed in a future version.
Instructions for updating:
Switch to tf.estimator.Exporter and associated utilities.
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/task.py", line 98, in
model.train_and_evaluate(arguments)
File "/home/jupyter/training-data-analyst/courses/machine_learning/deepdive2/recommendation_systems/labs/walsmodel/model.py", line 198, in train_and_evaluate
learn_runner.run(experiment_fn = experiment_fn, output_dir = args["output_dir"])
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/learn_runner.py", line 225, in run
return _execute_schedule(experiment, schedule)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/learn_runner.py", line 52, in _execute_schedule
return task()
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/experiment.py", line 688, in train_and_evaluate
export_results = self._maybe_export(eval_result)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/experiment.py", line 809, in _maybe_export
eval_result=eval_result))
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/export_strategy.py", line 110, in export
return self.export_fn(estimator, export_path, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/utils/saved_model_export_utils.py", line 479, in export_fn
strip_default_attrs=strip_default_attrs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/estimators/estimator.py", line 1400, in export_savedmodel
model_fn_lib.ModeKeys.INFER)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/learn/python/learn/estimators/estimator.py", line 1228, in _call_model_fn
model_fn_results = self._model_fn(features, labels, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/factorization/python/ops/wals.py", line 338, in _wals_factorization_model_function
get_col_projection)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/control_flow_ops.py", line 1224, in cond
orig_res_t, res_t = context_t.BuildCondBranch(true_fn)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/control_flow_ops.py", line 1061, in BuildCondBranch
original_result = fn()
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/factorization/python/ops/wals.py", line 326, in get_row_projection
transpose_input=False)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/factorization/python/ops/factorization_ops.py", line 773, in project_row_factors
row_weights=projection_weights)[0]
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/factorization/python/ops/factorization_ops.py", line 934, in _process_input_helper
lambda: math_ops.cast(row_weights, dtypes.float32))
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/control_flow_ops.py", line 1224, in cond
orig_res_t, res_t = context_t.BuildCondBranch(true_fn)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/control_flow_ops.py", line 1061, in BuildCondBranch
original_result = fn()
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/contrib/factorization/python/ops/factorization_ops.py", line 933, in
lambda: (array_ops.ones([num_indices]) * row_weights),
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py", line 2560, in ones
output = _constant_if_small(one, shape, dtype, name)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py", line 2295, in _constant_if_small
if np.prod(shape) < 1000:
File "<__array_function__ internals>", line 6, in prod
File "/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 3052, in prod
keepdims=keepdims, initial=initial, where=where)
File "/opt/conda/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 736, in __array__
" array.".format(self.name))
NotImplementedError: Cannot convert a symbolic Tensor (cond_1/strided_slice_4:0) to a numpy array.
---------------------------------------------------------------------------
CalledProcessError Traceback (most recent call last)
/tmp/ipykernel_19794/806600591.py in
----> 1 get_ipython().run_cell_magic('bash', '', 'rm -rf wals.tar.gz wals_trained\ngcloud ai-platform local train \\\n --module-name=walsmodel.task \\\n --package-path=${PWD}/walsmodel \\\n -- \\\n --output_dir=${PWD}/wals_trained \\\n --input_path=${PWD}/data \\\n --num_epochs=0.01 --nitems=${NITEMS} --nusers=${NUSERS} \\\n --job-dir=./tmp\n')

/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
2470 with self.builtin_trap:
2471 args = (magic_arg_s, cell)
-> 2472 result = fn(*args, **kwargs)
2473 return result
2474

/opt/conda/lib/python3.7/site-packages/IPython/core/magics/script.py in named_script_magic(line, cell)
140 else:
141 line = script
--> 142 return self.shebang(line, cell)
143
144 # write a basic docstring:

/opt/conda/lib/python3.7/site-packages/decorator.py in fun(*args, **kw)
230 if not kwsyntax:
231 args, kw = fix(args, kw, sig)
--> 232 return caller(func, *(extras + args), **kw)
233 fun.__name__ = func.__name__
234 fun.__doc__ = func.__doc__

/opt/conda/lib/python3.7/site-packages/IPython/core/magic.py in (f, *a, **k)
185 # but it's overkill for just that one bit of state.
186 def magic_deco(arg):
--> 187 call = lambda f, *a, **k: f(*a, **k)
188
189 if callable(arg):

/opt/conda/lib/python3.7/site-packages/IPython/core/magics/script.py in shebang(self, line, cell)
243 sys.stderr.flush()
244 if args.raise_error and p.returncode!=0:
--> 245 raise CalledProcessError(p.returncode, cell, output=out, stderr=err)
246
247 def _run_script(self, p, cell, to_close):

CalledProcessError: Command 'b'rm -rf wals.tar.gz wals_trained\ngcloud ai-platform local train \\\n --module-name=walsmodel.task \\\n --package-path=${PWD}/walsmodel \\\n -- \\\n --output_dir=${PWD}/wals_trained \\\n --input_path=${PWD}/data \\\n --num_epochs=0.01 --nitems=${NITEMS} --nusers=${NUSERS} \\\n --job-dir=./tmp\n'' returned non-zero exit status 1.
```

This is the second recommendation lab which fails, it needs revision for newer TF. Unfortunately I'm way to busy with work and trying to go through the course material to rewrite these notebooks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.