dask / dask/distributed

Keras serializer needs to recompile model for more training

Open
#2,183 8 comments 0 reactions 0 assignees View on GitHub
help wanted
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

In the [Keras FAQ](https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model),

> You can use model.save(filepath) to save a Keras model into a single HDF5 file which will contain:
>
> * the architecture of the model, allowing to re-create the model
the weights of the model
> * the training configuration (loss, optimizer)
> * the state of the optimizer, allowing to resume training exactly where you left off.

[`serialize_keras_model`][1] serializes the first two points, the weights and architecture of the model.
It does not serialize the optimizer or it's state.

A brief example:

``` python
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPooling2D
from keras import backend as K

model, x_train, y_train = model_and_data() # defined below in "➤ Details" tag
opt = keras.optimizers.Adam()
assert not hasattr(model, 'loss')
model.compile(loss='categorical_crossentropy',
optimizer=opt)
assert model.loss == 'categorical_crossentropy'

model.fit(x_train, y_train,
batch_size=128,
epochs=1,
verbose=1)

m2 = deserialize(*serialize(model))
assert not hasattr(m2, 'loss')

m2.fit(X, X)
```

which raises a Runtime error because the model hasn't been compiled:

``` python-traceback
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
in ()
----> 1 m2.fit(X, X)

/Users/ssievert/anaconda3/envs/dask-master/lib/python3.6/site-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
953 sample_weight=sample_weight,
954 class_weight=class_weight,
--> 955 batch_size=batch_size)
956 # Prepare validation data.
957 do_validation = False

/Users/ssievert/anaconda3/envs/dask-master/lib/python3.6/site-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
678 if y is not None:
679 if not self.optimizer:
--> 680 raise RuntimeError('You must compile a model before '
681 'training/testing. '
682 'Use `model.compile(optimizer, loss)`.')

RuntimeError: You must compile a model before training/testing. Use `model.compile(optimizer, loss)`.
```

Here's the definition of `model_and_data`:

def model_and_data():
img_rows, img_cols = 28, 28
(x_train, y_train), (x_test, y_test) = mnist.load_data()

x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
input_shape = (img_rows, img_cols, 1)

x_train = x_train.astype('float32') / 255
x_test = x_test.astype('float32') / 255
y_train = keras.utils.to_categorical(y_train, 10)
y_test = keras.utils.to_categorical(y_test, 10)

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))
return model, x_train, y_train

[1]:serialize_keras_model

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.