ageron / ageron/handson-ml2

[IDEA] Use of TextVectorization layer for sentimental analysis in chapter 16

Open
#441 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
30k
Forks
13.1k
PR merge metrics
No merged PRs in 30d

Description

The following code snippet illustrates preprocessing and encoding of texts and use the result to train GRU models for sentimental analysis in chapter 16.

```python
embed_size = 128
model = keras.models.Sequential([
keras.layers.Embedding(vocab_size + num_oov_buckets, embed_size,
mask_zero=True, # not shown in the book
input_shape=[None]),
keras.layers.GRU(128, return_sequences=True),
keras.layers.GRU(128),
keras.layers.Dense(1, activation="sigmoid")
])
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
history = model.fit(train_set, steps_per_epoch=train_size // 32, epochs=5)
```

It would be great to add how to use keras.layers.experimental.preprocessing.TextVectorization for the same goal. For that, I tried the following code snippet. But the problem in this case, the training becomes too slow:

```python
from tensorflow.keras.layers.experimental.preprocessing import TextVectorization

vectorize_layer = TextVectorization(max_tokens=vocab_size+1, output_mode='int')
vectorize_layer.adapt(truncated_vocabulary)

embed_size = 128
model = keras.models.Sequential([
vectorize_layer,
keras.layers.Embedding(vocab_size + 1, embed_size,
mask_zero=True, # not shown in the book
input_shape=[None]),
keras.layers.GRU(128, return_sequences=True),
keras.layers.GRU(128),
keras.layers.Dense(1, activation="sigmoid")
])
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
history = model.fit(datasets["train"].batch(32).prefetch(1), steps_per_epoch=train_size // 32, epochs=5)
```

Any advice please?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.