ageron / ageron/handson-ml2

[QUESTION]CH16, Sentiment Analysis Numpy version issue/Memory leak?

Open
#419 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
30k
Forks
13.1k
PR merge metrics
No merged PRs in 30d

Description

I found couple of issues executing this code, while using local environment to execute the code.
Well this was the code from the Textbook, and notebook file of ch 16, from Sentiment Analysis.

```python
import tensorflow_datasets as tfds
import tensorflow as tf
from tensorflow import keras
import tensorboard
import os
import numpy as np

(X_train, y_train), (X_test, y_test) = keras.datasets.imdb.load_data()

X_train[0][:10]

datasets, info = tfds.load("imdb_reviews", as_supervised=True, with_info=True)
train_size = info.splits["train"].num_examples
test_size = info.splits["test"].num_examples
print(train_size)

X_train[0][:10]

word_index = keras.datasets.imdb.get_word_index()
id_to_word = {id_ + 3: word for word, id_ in word_index.items()}
for id_, token in enumerate(("", "", "")):
id_to_word[id_] = token
" ".join([id_to_word[id_] for id_ in X_train[0][:10]])

import tensorflow_datasets as tfds

datasets, info = tfds.load("imdb_reviews", as_supervised=True, with_info=True)

datasets.keys()

train_size = info.splits["train"].num_examples
test_size = info.splits["test"].num_examples

for X_batch, y_batch in datasets["train"].batch(2).take(1):
for review, label in zip(X_batch.numpy(), y_batch.numpy()):
print("Review:", review.decode("utf-8")[:200], "...")
print("Label:", label, "= Positive" if label else "= Negative")
print()

def preprocess(X_batch, y_batch):
X_batch = tf.strings.substr(X_batch, 0, 300)
X_batch = tf.strings.regex_replace(X_batch, rb"", b" ")
X_batch = tf.strings.regex_replace(X_batch, b"[^a-zA-Z']", b" ")
X_batch = tf.strings.split(X_batch)
return X_batch.to_tensor(default_value=b""), y_batch

preprocess(X_batch, y_batch)

from collections import Counter

vocabulary = Counter()
for X_batch, y_batch in datasets["train"].batch(32).map(preprocess):
for review in X_batch:
vocabulary.update(list(review.numpy()))

vocab_size = 10000
truncated_vocabulary = [
word for word, count in vocabulary.most_common()[:vocab_size]]

words = tf.constant(truncated_vocabulary)
word_ids = tf.range(len(truncated_vocabulary), dtype=tf.int64)
vocab_init = tf.lookup.KeyValueTensorInitializer(words, word_ids)
num_oov_buckets = 1000
table = tf.lookup.StaticVocabularyTable(vocab_init, num_oov_buckets)

def encode_words(X_batch, y_batch):
return table.lookup(X_batch), y_batch

train_set = datasets["train"].repeat().batch(32).map(preprocess)
train_set = train_set.map(encode_words).prefetch(1)

embed_size = 128
model = keras.models.Sequential([
keras.layers.Embedding(vocab_size + num_oov_buckets, embed_size,
mask_zero=True, # not shown in the book
input_shape=[None]),
keras.layers.GRU(128, return_sequences=True),
keras.layers.GRU(128),
keras.layers.Dense(1, activation="sigmoid")
])
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
history = model.fit(train_set, steps_per_epoch=train_size // 32, epochs=5)
```

First, Numpy 1.20.1 does not work with TF 2.4.1 in this code.
it will return

`Cannot convert a symbolic Tensor (gru/strided_slice:0) to a numpy array.`

as a exception message, I'm sorry that I could not get full message since I solved it by
downgrading the numpy to 1.19.2.

`pip install numpy==1.19.2`

will handle this problem anyway.

Second, even if we solved the version issue, we still have memory issue.

```
[_Derived_]RecvAsync is cancelled.
[[{{node Adam/Adam/update/AssignSubVariableOp/_41}}]]
[[gradient_tape/sequential/embedding/embedding_lookup/Reshape/_38]] [Op:__inference_train_function_115627]
```
For this exception message, I added

```
gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
try:
tf.config.experimental.set_memory_growth(gpus[0], True)
except RuntimeError as e:


print(e)
```
to solve the problem. However, I discovered that local GPU memories are not utilized perfectly.
I'll attach screenshots for Performance graph from task manager of windows 10.
### This is when I run other codes from textbook
![Usual case](https://user-images.githubusercontent.com/56079922/115705939-1a89be00-a3a8-11eb-9ff7-2dd4aaefbb49.PNG)

### Aand this is what happens when I run the code that matters:/
![Ch16Sentiment](https://user-images.githubusercontent.com/56079922/115705960-1f4e7200-a3a8-11eb-9683-b956c9881389.PNG)

Does anyone know why are they happening?
Btw, sorry for screenshots including some Korean.
hope you guys understand the screenshot with context:)

**Versions (please complete the following information):**
- OS: WIN10
- Python: 3.8.8
- TensorFlow: 2.4.1
- Numpy: 1.20.1->1.19.2
- GTX1660TI, CUDA11
-

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.